Skip to content

fix(validation): honor segment sample percentage on large releases (#812) - #817

Merged
javi11 merged 1 commit into
mainfrom
session/altmount-issue-812-60fcbd
Aug 10, 2026
Merged

fix(validation): honor segment sample percentage on large releases (#812)#817
javi11 merged 1 commit into
mainfrom
session/altmount-issue-812-60fcbd

Conversation

@javi11

@javi11 javi11 commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Fixes #812.

Problem

The segment scan percentage setting had no effect below 100%. Sampling applied a hard 55-sample ceiling after the percentage math, so on any release with more than ~1,100 segments every setting from 1% to 99% collapsed onto the same 55 STATs. Only 100% behaved as configured, because it takes an early-return path that skips sampling entirely — which is exactly the asymmetry the reporter observed.

The UI presents this as a plain percentage slider ("1% (FAST)" → "100% (SLOW)") with no mention of a cap, so the ceiling was silently overriding an explicit user choice.

Changes

  • internal/usenet/validation.goselectSegmentsForValidation (health checks; the path the settings slider drives) drops min(..., 55) and honors the configured percentage exactly. The min-5 floor for statistical validity is kept.
  • internal/importer/validation/fast_fail.goselectFastFailSegments (import fast-fail, per file) had the same maxSamples = 55 ceiling, which capped even 100% sampling. It now takes first + last + exactly pct% of the middle; the redundant clamp is removed and slice capacity is sized from the real sample count.
  • Stale ≤55 comments in processor.go and the FastFailReleaseProbe doc updated.

Reviewer notes

This raises default sampling cost. The 55 cap was doing real work to keep out-of-box checks cheap. At the 5% health default, a ~7,000-segment release moves from 55 STATs to ~350 per check; the 1% import default is milder. That is the intended meaning of the setting, but if it proves too aggressive in practice, lowering the shipped default in manager.go:1663 is a one-line follow-up. The alternative considered was making the cap a new config knob — rejected as extra surface for a setting that should already mean what it says.

Testing

The two tests that asserted the 55 cap are rewritten as #812 regressions, and were confirmed red against the old code (got 55, want 2000) before the fix:

  • health sampling: 20,000 segments → 2,000 at 10%, 10,000 at 50%, strictly monotonic across percentages.
  • fast-fail sampling: 10,000 at 100%, 1,002 at 10%, no duplicates.

go build ./..., go vet ./internal/... and the full go test ./... pass; internal/usenet, internal/importer, internal/importer/validation and internal/health re-verified with -race.

)

Segment sampling applied a hard 55-sample ceiling after the percentage
math, so on any release with more than ~1,100 segments every setting from
1% to 99% collapsed onto the same 55 STATs. Only 100% behaved as
configured, because it takes an early-return path that skips sampling
entirely.

Remove the ceiling in both sampling paths:

- selectSegmentsForValidation (health checks, the settings slider) now
  honors the configured percentage exactly, keeping the min-5 floor for
  statistical validity.
- selectFastFailSegments (import fast-fail, per file) had the same
  maxSamples = 55 ceiling, which capped even 100% sampling. It now takes
  first + last + exactly pct% of the middle; the redundant clamp is gone
  and slice capacity is sized from the real sample count.

The two tests that asserted the cap are rewritten as regressions.

Note: this raises default sampling cost. At the 5% health default a
~7,000-segment release moves from 55 STATs to ~350 per check; the 1%
import default is milder.
@javi11
javi11 merged commit a789968 into main Aug 10, 2026
2 checks passed
@javi11
javi11 deleted the session/altmount-issue-812-60fcbd branch August 10, 2026 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Segment scan percentage not working - v0.3.2

1 participant