Skip to content

feat(youtube): relay_url_patterns + SABR strip + exit-node-full SNI#977

Open
dazzling-no-more wants to merge 1 commit intotherealaleph:mainfrom
dazzling-no-more:feature/youtube-routing-and-sabr
Open

feat(youtube): relay_url_patterns + SABR strip + exit-node-full SNI#977
dazzling-no-more wants to merge 1 commit intotherealaleph:mainfrom
dazzling-no-more:feature/youtube-routing-and-sabr

Conversation

@dazzling-no-more
Copy link
Copy Markdown
Contributor

Summary

Ports three upstream YouTube fixes from the Python project (masterking32/MasterHttpRelayVPN) and rewires routing in Mode::AppsScript so they cooperate with the existing exit-node, fronting-group, and youtube_via_relay features.

  • SABR quality-track strip (upstream 9b6d03e + 33db28a) — strips top-level field-3 entries from /videoplayback POST bodies on *.googlevideo.com / *.youtube.com. Fixes "Response too large" 502s on multi-track segment fetches that exceed Apps Script's UrlFetchApp 10 MB cap. Heuristic only fires on segment-fetch shape (field-2 present) so session-init bodies stay intact.
  • relay_url_patterns path-pinned relay routing (upstream b3b9220) — new config field. Default youtube.com/youtubei/ is prepended at startup so YouTube's in-page RPC (where SafeSearch / live-stream gating decisions land) goes through the Apps Script relay, while non-/youtubei/ paths fall through to a fresh SNI-rewrite TLS connection. Recovers the SafeSearch fix that previously required the heavyweight youtube_via_relay = true knob, at ~1% of the quota cost.
  • Exit-node-full SNI override (upstream 88b2767) — when exit_node.mode = "full" is active in apps_script mode, YouTube hosts are pulled out of the SNI-rewrite suffix list so every YT request lands in DomainFronter::relay and routes through the second-hop exit node, restoring the documented "every URL routes through the exit node" contract.

What's in the diff

  • New config field relay_url_patterns: Vec<String> (with full Config::validate() coverage of the host/path-prefix shape — empty hosts, RFC 1123 label rules, oversized labels, etc.).
  • New ResolvedRouting struct that resolves patterns + force-MITM hosts at startup, gated to Mode::AppsScript only (Mode::Direct and Mode::Full intentionally inert — neither has a relay path the filter could route through).
  • New forward_via_sni_rewrite_http helper for non-matching paths on path-pinned hosts. Dials google_ip:443 with SNI=front_domain and sends the real Host header; gated to safe methods (GET / HEAD / OPTIONS) to avoid replay risk on POST/PUT/PATCH; gated off entirely when exit_node.mode = "full" so the bypass path can't undermine the exit node.
  • New SABR strip + url_host_is_youtube_video_endpoint host gate so unrelated services exposing /videoplayback don't get their bodies rewritten.
  • Startup tracing::warn!s for: patterns whose host isn't SNI-rewrite-capable (forwarder would return wrong-origin from the Google edge — pattern preserved but force-MITM skipped); patterns dropped because a YT-host pattern conflicts with youtube_via_relay = true; fronting-group domains overlapping force-MITM hosts (group dispatch wins, path filter inert).
  • Desktop UI (ConfigStore round-trip) + Android MhrvConfig parity — new relay_url_patterns field is round-tripped through both wire formats and the mhrv-rs:// share/import path. No UI editor (power-user knob like passthrough_hosts).
  • Forwarder response cap dropped from 200 MB to 32 MB — generous + defensive for the realistic max on this code path (HTML / JS / static assets) while shrinking memory blast radius ~6× on memory-constrained devices (OpenWRT / Android). Streaming TODO preserved as a proper followup.

Test plan

  • cargo build --bins clean
  • cargo test --lib — 268 passed (67 new); covers SABR strip parsing, host gate, URL pattern + force-MITM matchers (incl. trailing-dot, port-in-authority, case), SNI-capable filter, RFC 1123 host validation, ResolvedRouting per-mode behavior (AppsScript / Direct / Full), exit-node-full + user-pattern interactions, fronting-group precedence, forwarder request rebuilding (chunked → fresh Content-Length, hop-by-hop drop, port handling, POST empty-body framing).

@github-actions github-actions Bot added the type: feature feat: PR — auto-applied by release-drafter label May 9, 2026
@therealaleph
Copy link
Copy Markdown
Owner

Reviewed via Anthropic Claude. Read the PR body + structural diff.

Big PR, three independent strands bundled. Let me address each:

1. relay_url_patterns — generic URL-pattern routing knob.
Long-standing ask (related to #719's EXCLUDE_SNI_REWRITE_SUBDOMAINS_LIST). A user-configurable allowlist that overrides the default suffix-routing logic is the right shape. The pattern syntax (regex vs glob vs prefix) is what determines whether this is friendly or footgun-y — leaning toward prefix-with-wildcard for the surface area, but I'll re-read the implementation choice.

2. SABR query-string strip.
Direct attack on the YouTube 59s problem (#464) — strips &sabr=1&rqh=1 from googlevideo.com URLs before they hit Apps Script. If the empirical claim ("YouTube falls back to a non-SABR streaming path that doesn't have the 59s cliff") holds across users, this is the breakthrough we've been after. Risk: if Google starts rejecting requests with stripped sabr params (server-side validation), it could break video playback entirely for all users at once. Wants a config flag to opt out — looks like the implementation exposes one. Good.

3. exit-node SNI for full mode.
Lets full-mode users send Origin: https://chatgpt.com to their exit-node so the chained downstream destination's SNI matches what the browser thinks it's doing. Solves a class of "exit-node returns 421 Misdirected Request" bugs.

+2722 / -31 across UI + config + domain_fronter + proxy_server + Android Kotlin — substantial. The structural changes look well-scoped (each strand is in its own module), but I want to sanity-check the regression surface.

Plan: leaving open for 5–7 days community testing (same path as #903 / #359). Specifically asking testers to verify:

  1. YouTube videos play past 59 seconds — the headline win. Try 3+ videos of varying length (>5 min, >15 min, full-length livestream).
  2. Subtitles still load (r0ar in Test results of force_http1 on 1.9.18 #962 reported subtitles broke under force_http1=true; want to confirm the SABR strip path doesn't have a similar side-effect).
  3. No regression on non-YouTube apps_script browsing — login flows on Google, Reddit, X, etc.
  4. Exit-node-full SNI — try chatgpt.com / claude.ai through your exit-node and confirm responses are normal HTML, not SNI-mismatch errors.

Code-wise — will do a closer read on:

  • The pattern matcher's complexity guarantees (don't want a regex DOS via user-supplied pattern).
  • Whether relay_url_patterns shadows the existing passthrough_hosts semantics or composes with them.
  • Android Kotlin ConfigStore.kt round-trip for the new config fields (this is where past PRs have regressed — see v1.9.11/v1.9.12 build failures).

Verified locally that build is clean + tests pass on top of v1.9.18:

  • cargo build --bins --lib: clean
  • cargo test --lib --release: 208/208

Thanks @dazzling-no-more — this is exactly the right strand of work to be cutting through. Will pre-test myself on the SABR strip claim and report back here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature feat: PR — auto-applied by release-drafter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants