Skip to content

perf: cache CryptoProvider/RootCertStore/BrowserEmulator, optimize Python iterators, fix Random OS consistency#148

Merged
deedy5 merged 3 commits into
mainfrom
dev
May 19, 2026
Merged

perf: cache CryptoProvider/RootCertStore/BrowserEmulator, optimize Python iterators, fix Random OS consistency#148
deedy5 merged 3 commits into
mainfrom
dev

Conversation

@deedy5

@deedy5 deedy5 commented May 19, 2026

Copy link
Copy Markdown
Owner

Summary:

  1. 81f4e35 — perf: cache TLS config, eliminate recursion
  • Cache CryptoProvider and RootCertStore in OnceLock to avoid repeated lookups and allocations
  • Cache BrowserEmulator in Arc via OnceLock across all browsers (Chrome, Edge, Firefox, Opera, Safari) — avoids cloning TLS fingerprint vectors on every request
  • Eliminate recursion in os_platform() and Safari's build_user_agent() by resolving Random upfront
  • Set os_type on Safari BrowserEmulator via Arc::make_mut
  1. b75c8fd — perf: optimize Python iterators, reduce clones
  • Cache encoding lookup per-iterator (&'static Encoding) instead of per-chunk in TextIterator/AsyncTextIterator
  • Pre-allocate buffers with Vec::with_capacity()
  • Change LinesIterator/AsyncLinesIterator buffer from String to Vec — avoids UTF-8 conversion on every chunk, only converts on line extraction
  • Move redirect policy restoration before async block (reduces lock-hold window)
  • Eliminate headers.clone() in all Python HTTP method helpers (pass ownership)
  1. 58215c5 — fix: consistent Random OS resolution
  • Resolve ImpersonateOS::Random once per build_*_settings call instead of independently at each consumer (User-Agent, sec-ch-ua-mobile, sec-ch-ua-platform, Safari TLS os_type), preventing header inconsistencies

deedy5 added 3 commits May 19, 2026 13:29
…> in OnceLock, eliminate recursion in os_platform/build_user_agent
…istent headers

Previously each consumer (build_user_agent, os_platform, sec-ch-ua-mobile)
independently generated a random OS when Random was selected, leading to
inconsistent headers (e.g. User-Agent saying Linux but sec-ch-ua-platform
saying Windows). Now Random is resolved to a concrete OS once at the top
of each build_*_settings function, ensuring all consumers agree.
@deedy5 deedy5 merged commit 420b4a1 into main May 19, 2026
36 checks passed
@deedy5 deedy5 deleted the dev branch May 19, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant