fix(sentry): drop Redis connect-time failures from the container-restart race - #134
Merged
Merged
Conversation
…art race Sentry #957 accumulated 808 events since 2026-07-01 and buried the rest of the fleetq project. It is not a chronic outage: the 100 most recent events fall into three bursts (2026-07-21, -07-22, -08-03) with days of silence between, each lining up with a container recreate / redis config change. Culprit is /var/www/artisan — scheduler and queue workers reconnecting while redis is still coming up. Same shape as the existing SQLSTATE[08006] postgres filter, for the other datastore. Scoped to Predis StreamInitException, which is thrown only from StreamFactory during stream init, so a read/write error on an established connection still reports (regression-tested). Also drops the prometheus client's own wrapper of the same failure reaching the /metrics scrape (#1082), which the class check cannot catch. A sustained outage stays visible: HealthController pings Redis and flips /api/v1/health to 503 degraded.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the noise behind Sentry issue #957 (
Predis\Connection\Resource\Exception\StreamInitException: Connection refused [tcp://agent-fleet-redis:6379]) and #1082 (prometheus client wrapper of the same failure).Why this is noise, not an outage
957 has 808 events since 2026-07-01 and was burying the fleetq project. It is bursty, not chronic — the 100 most recent events fall into three windows:
Days of silence in between. Each burst lines up with a container recreate or redis config change (prod carries a
.env.bak-redis-20260722075736from the 07-22 window; the 08-03 pair is the deploy of #133). Culprit is/var/www/artisan— scheduler and queue workers reconnecting while redis is still coming up.This is the same failure mode the filter already suppresses for Postgres one branch above:
// SQLSTATE[08006] — postgres connection_failure on container restart race.Scope — what still reports
Deliberately narrow, because blanket-suppressing Redis errors would hide real defects:
StreamInitExceptiononly. Predis throws it exclusively fromStreamFactoryduring stream init, i.e. connect time. A read/write error on an already-established connection (the class fixed in16c2b9f2) is untouched — there is a regression test asserting exactly that.HealthControllerpings Redis and flips/api/v1/healthto503 degraded. Verified in the controller, not assumed.instanceofcheck cannot catch.Tests
3 added to
BeforeSendFilterWebhookTest: dropsStreamInitException, drops the prometheus wrapper, and keeps aConnectionExceptionread error. 9 passed. Pint + phpstan clean on the touched paths.