feat(sabnzbd): paginate history via a single UNION query (no schema change) - #788
Merged
Merged
Conversation
…hange) The mode=history handler merged completed-queue + a 7-day import_history window + failed-queue and paginated in memory over hardcoded caps (2000/1000), so clients couldn't page past the caps and noofslots was wrong. Move the merge + dedup + pagination into one SQL query over the existing tables — no migration, no schema change: - add ListSABnzbdHistory + CountSABnzbdHistory (UNION ALL of import_queue[completed] + import_history + import_queue[failed]), deduped by the import_history.nzb_id = import_queue.id anti-join so a completed import present in both tables is counted once; ORDER BY completed_at DESC, id DESC with real LIMIT/OFFSET and an exact COUNT - rewrite the handler bulk path to use them; cap limit at 1000 - #596 preserved: only rows sourced from the live completed queue are flipped to Failed on a missing path - nzo_ids reconciliation path and failed-item retention are unchanged Failed items keep living in import_queue with their existing FailedItemRetentionHours lifecycle; nothing about failures or retention changes. Adds repository tests for dedup, sources, pagination and category.
javi11
force-pushed
the
session/sabnzbd-history-pagination-8e7819
branch
from
July 20, 2026 11:20
e9798bf to
3a30266
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fix pagination of the SABnzbd-compatible
mode=historyAPI by moving the merge + dedup + pagination into one SQL query over the existing tables. No schema change, no migration.Why
The handler merged three sources — completed rows from
import_queue, a 7-day window ofimport_history, and failed rows fromimport_queue— deduped and sliced them in memory over hardcoded caps (2000 / 1000). So clients (Sonarr/Radarr) could never page past the caps, andnoofslotsdidn't reflect the true total.How
ListSABnzbdHistory+CountSABnzbdHistory: aUNION ALLofimport_queue(completed) +import_history+import_queue(failed), deduped by an anti-join onimport_history.nzb_id = import_queue.id(a completed import can live in both tables — the live queue row wins, the history copy is dropped),ORDER BY completed_at DESC, id DESC, realLIMIT/OFFSET, and an exactCOUNT. Portable across SQLite + Postgres (no window functions, no basename/regexp;?andFALSEhandled by the dialect layer).limitcapped at 1000.Failedwhen their reported path is missing on disk.nzo_idsreconciliation path unchanged.What does NOT change
import_queuewith their existingFailedItemRetentionHourslifecycle (default 24h) — same as before.Behavior note
Dedup shifts from filename-based to queue-id-based (
import_history.nzb_id). This is more correct than the old name dedup (which was partly ineffective because queue paths carry a{id}-prefix thatimport_history.nzb_namestrips): a completed import in both tables now reliably shows once, and two genuinely distinct jobs that share a filename both show (as real SABnzbd does). The old fixed history window (7 days) is gone — pagination now spans the full history.Test plan
go test ./internal/database/... ./internal/api/...— green;go vetclean;go build ./...cleanstartbeyond totalmode=historywithstart/limit(no page overlap, constantnoofslotsacross pages)