Skip to content

chore(river): modularity + readability cleanup across the River systems#400

Merged
jiashuoz merged 3 commits into
mainfrom
chore/river-cleanup
Jul 9, 2026
Merged

chore(river): modularity + readability cleanup across the River systems#400
jiashuoz merged 3 commits into
mainfrom
chore/river-cleanup

Conversation

@jiashuoz

@jiashuoz jiashuoz commented Jul 9, 2026

Copy link
Copy Markdown
Member

Readability/modularity pass across the River job systems, driven by three parallel code-review agents (cross-system interface audit + two per-package reviews). No behavior change — pure cleanup, verified by each package's existing tests. Net −124 lines.

The reviews' headline conclusion: the River systems are already well-disciplined and consistent (uniform Registrar + two-phase SetEnqueuer, principled queue routing, consumer-side narrow interfaces). These are the polish items that survived scrutiny.

1. Hoist the copy-pasted reconciler → jobs.ReconcilePending (biggest win)

The stranded-row reconciler was duplicated across 5 packages (~240 lines) with byte-identical control flow — scan WHERE <pred> AND <job_col> IS NULL → per-row FOR UPDATE re-check → enqueue → stamp. Only the table/predicate/job-column/log-prefix varied. Extracted a single jobs.ReconcilePending(pool, spec, enqueueTx) + ReconcileSpec + DefaultReconcileBatch; each package collapses to a spec literal + one call. Behavior-identical (the two store-method stampers did the exact same UPDATE the helper now inlines; those methods stay for the accept-tx path). Domain method signatures unchanged, so every caller is untouched. The reconcile invariants now live in one hot-path-tested place instead of five.

2. webhookpub dead code + docstring drift

  • Deleted committed AI-scratchpad text + 3 no-op var _ compile-guards in worker.go (and the 3 imports they alone kept), the unused nowUTC, and dead oldest telemetry scaffolding.
  • Fixed docstring drift: writeOutboxRow's "(eventually) PublishBestEffortTx" (already wired) + the pg_notify block now notes it's a harmless no-op in river mode; PublishBestEffortTx's reference to a webhook_publish_failures table that was never built + a "legacy-fallback" line contradicting the file itself.
  • Rewrote the event.go package header (it still described the retired in-process goroutine publisher) to the current three-layer outbox→fan-out→River pipeline.
  • Dropped stale internal "slice-N" markers; standardized the fan-out Kind() webhook_fan_outwebhook_fanout (safe — flag defaults legacy, no such jobs in prod yet).

3. Timeout documentation + small consistency polish

  • Documented the deliberate reliance on River's 60s default JobTimeout on the workers that don't override Timeout() — so the omission reads as a decision, not an oversight (the repo has an explicit "JobTimeout trap" lesson). InboundProcessWorker notes the content screen is bounded by piguard's 5s per-detector timeout.
  • jobs.New: defaultQueueConfig(cfg) instead of six positional ints (transposition hazard).
  • RegisterJobs now calls the NewMaintenanceWorker constructors instead of rebuilding the struct literal (they exist precisely so tests build an identical worker).
  • Named the markFailed terminal-write retry magic numbers; "legacy publisher" → "webhook publisher" in hitlworker.

Explicitly left alone (reviews confirmed these are correct, not gaps)

  • The three enqueuer seams (EnqueueSendTx/EnqueueDeliveryTx/EnqueueFanOutTx) stay separate — domain-meaningful names, textbook consumer-side ISP.
  • Fan-out co-located in webhookpub (shares fanOutEventCore with the outbox), delivery in webhookdelivery — the split tracks real coupling.
  • Two-phase wiring, live-reconcile-vs-cutover split, queue routing — all uniform and principled.

Noted follow-ups (not in this PR)

🤖 Generated with Claude Code

jiashuoz and others added 3 commits July 9, 2026 12:20
Readability pass on webhookpub (no behavior change):
- Delete committed AI-scratchpad text + 3 no-op `var _` compile guards in
  worker.go, and the now-unused errors/encoding-json/pgconn imports.
- Delete unused `nowUTC` var (+ its time import) in outbox.go.
- Delete the dead `oldest` telemetry scaffolding in Tick (declared, ranged
  into nothing, discarded).
- Fix docstring drift: writeOutboxRow's "(eventually) PublishBestEffortTx"
  (already wired) + acknowledge that in river mode nothing LISTENs the
  NOTIFY; PublishBestEffortTx's reference to a webhook_publish_failures
  table that was never built + the "legacy-fallback" line that contradicts
  the file's own "no legacy fallback path left"; the pg_notify block now
  notes it's a harmless no-op in river mode.
- Rewrite the event.go package header to describe the current durable
  three-layer outbox→fan-out→River pipeline (it still described the retired
  in-process post-commit-goroutine publisher + a tmp/ design path).
- Drop stale internal "slice-N" markers; note both fan-out engines share
  fanOutEventCore in the matches/Publisher docs.
- Standardize the fan-out job Kind() "webhook_fan_out" → "webhook_fanout"
  to match webhook_fanout_reconcile + the rest of the "fanout" naming
  (safe: E2A_WEBHOOK_FANOUT_MODE defaults legacy, so no such jobs exist in
  prod yet).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfAXBRMJ3KPuU3UEgwqjhS
…ending

The stranded-row reconciler was duplicated across five packages
(outboundsend, inboundprocess, hitlnotify, webhookdelivery, webhookpub
fan-out) — ~240 lines with byte-identical control flow: scan
`WHERE <pred> AND <job_col> IS NULL LIMIT N` → per-row `FOR UPDATE`
re-check → skip-if-set → EnqueueXTx → stamp the job id. Only the
table/predicate/job-column/log-prefix varied.

Extract a single `jobs.ReconcilePending(pool, spec, enqueueTx)` +
`jobs.ReconcileSpec` + `jobs.DefaultReconcileBatch`. Each package's
ReconcilePending collapses to a spec literal + one call; the domain
method signatures are unchanged, so every caller (main.go cutover +
each ReconcileWorker) is untouched.

Behavior-identical: the two packages that stamped via a store method
(inboundprocess/hitlnotify) stamped with the exact same
`UPDATE <table> SET <job_col>=$2 WHERE id=$1` the helper now inlines;
those store methods stay for the accept-tx path. Log output (prefixes)
is preserved verbatim. The interpolated identifiers are compile-time
constants (documented in ReconcileSpec) — the row id is always a bound
param.

Net ~240 → ~40 shared lines, and the reconcile invariants (FOR UPDATE
re-check, IS NULL guard) now live in one hot-path-tested place instead
of five. Each package keeps its own reconcile_test.go — all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfAXBRMJ3KPuU3UEgwqjhS
Readability polish across the River worker packages (no behavior change),
from the modularity review:

- Document the deliberate reliance on River's 60s default JobTimeout on
  the four per-message workers that DON'T override Timeout() (SendWorker,
  DeliverWorker, InboundProcessWorker, NotifyWorker) + RetentionWorker —
  so the omission reads as a decision, not an oversight (the repo has an
  explicit "River JobTimeout trap" lesson). InboundProcessWorker notes the
  content screen is bounded by piguard's own 5s per-detector timeout;
  RetentionWorker notes its single unbounded DELETE is ctx-safe under a
  60s cut and flags batching (like #397) as a future option.
- jobs.New: pass Config to defaultQueueConfig(cfg) instead of six
  positional ints (removes an invisible-transposition hazard).
- Have RegisterJobs call the NewMaintenanceWorker constructors
  (webhookdelivery, hitlworker, janitor) instead of rebuilding the struct
  literal by hand — the constructors exist precisely so tests build an
  identical worker; now RegisterJobs can't drift from them.
- outboundsend: name the markFailed terminal-write retry magic numbers
  (terminalWriteRetries / terminalWriteBackoff); drop landed "(slice
  B/C/D)" internal markers from comments.
- hitlworker: "the legacy publisher" → "the webhook publisher" (the wired
  OutboxPublisher is the current outbox path, not the retired one).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DfAXBRMJ3KPuU3UEgwqjhS
@jiashuoz
jiashuoz merged commit 677116b into main Jul 9, 2026
17 checks passed
@jiashuoz
jiashuoz deleted the chore/river-cleanup branch July 9, 2026 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant