Skip to content

Add Sentry Logs observability and memory tracker middleware#215

Merged
JuanVqz merged 1 commit into
mainfrom
feature/sentry-memory-observability
May 6, 2026
Merged

Add Sentry Logs observability and memory tracker middleware#215
JuanVqz merged 1 commit into
mainfrom
feature/sentry-memory-observability

Conversation

@JuanVqz
Copy link
Copy Markdown
Member

@JuanVqz JuanVqz commented May 6, 2026

Context

Web dyno is hitting R14 continuously (average 114% quota, peak 209%). We are investigating the memory leak. This PR adds observability tooling to identify the source and reduces Sentry noise/memory pressure from the current capture_message approach.

Changes

capture_message -> Sentry.logger.info on all legacy traffic endpoints

All seven Sentry.capture_message calls replaced with Sentry.logger.info. Hits now land in Sentry Logs instead of firing Slack alerts and burning event quota. Extra debug fields added per endpoint to help answer who is hitting what:

Endpoint New fields
POST /gems method, path, format
GET /gems/:id format (crawler vs API consumer)
GET /gems/compat_table gemmy_count (spot bulk callers)
GET /gems/:gemmy_id/compatibility/:id renamed gemmy_id -> gem_name
POST /api/github_notifications check_name, status (full check_run state)
POST /api/results success field (was checker reporting success or failure?)
POST /api/releases renamed name -> gem_name for consistency

MemoryTracker rack middleware (new)

app/middleware/memory_tracker.rb reads /proc/$PID/status VmRSS before and after each request. When a single request grows RSS by 5+ MB it logs to Sentry.logger.warn with:

rss_before_mb, rss_after_mb, delta_mb, method, path, query_string, remote_ip, user_agent, referer

This will pinpoint exactly which endpoint and which request parameters are growing memory. The /proc read is Linux-only (Heroku dynos); the rescue fallback returns 0.0 so it is silent locally on macOS.

config/initializers/sentry.rb tuning

Three changes to reduce Sentry's own memory footprint:

  • Removed enabled_patches: [:logger] - was routing every Rails.logger call through Sentry's breadcrumb buffers, holding string references for the lifetime of each request
  • Added max_breadcrumbs: 20 - caps per-request breadcrumb buffer (default was 100)
  • Changed include_local_variables: false - was capturing all local variable values on every exception, which can hold large ActiveRecord objects in Sentry's internal buffers

Sentry gem version alignment

sentry-rails and sentry-sidekiq bumped from ~> 5.5 to ~> 6.5 to match sentry-ruby ~> 6.5. Having sentry-ruby 6.x with sentry-rails 5.x was a version mismatch.

- Swap capture_message to Sentry.logger.info on all legacy traffic
  endpoints so hits land in Sentry Logs instead of burning event quota
  and firing Slack alerts
- Add extra debug fields per endpoint (format, delta_mb, check_name,
  success, gemmy_count) to answer who/what is hitting each path
- Add MemoryTracker rack middleware: logs to Sentry.logger.warn when
  a single request grows RSS by 5+ MB, with full request context
- Tune sentry.rb: drop enabled_patches logger (was routing all Rails
  logs through Sentry breadcrumb buffers), cap max_breadcrumbs at 20,
  disable include_local_variables to avoid holding large AR objects
- Bump sentry-rails and sentry-sidekiq from ~> 5.5 to ~> 6.5 to
  match sentry-ruby ~> 6.5
@JuanVqz JuanVqz self-assigned this May 6, 2026
@JuanVqz JuanVqz requested a review from a team May 6, 2026 15:52
@JuanVqz JuanVqz marked this pull request as ready for review May 6, 2026 15:52
@JuanVqz JuanVqz merged commit 5c2ca81 into main May 6, 2026
2 checks passed
@JuanVqz JuanVqz deleted the feature/sentry-memory-observability branch May 25, 2026 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant