Skip to content

add category to OTel logs#1556

Merged
nitisht merged 3 commits intoparseablehq:mainfrom
nikhilsinhaparseable:log-category
Feb 25, 2026
Merged

add category to OTel logs#1556
nitisht merged 3 commits intoparseablehq:mainfrom
nikhilsinhaparseable:log-category

Conversation

@nikhilsinhaparseable
Copy link
Contributor

@nikhilsinhaparseable nikhilsinhaparseable commented Feb 25, 2026

derive a log category - FATAL, ERROR, WARN, INFO, DEBUG, TRACE for each ingested log records

first map severity_number directory to a category
fallback where severity_number is unset (0 / UNSPECIFIED),
scan the log body to find category based on case-insensitive substring match

defaults to UNSPECIFIED when neither rules yields a result

Summary by CodeRabbit

  • New Features

    • Logs now include automatic categorization based on severity and message content, assigning TRACE, DEBUG, INFO, WARN, ERROR, FATAL, or UNSPECIFIED.
  • Style

    • Minor formatting-only changes to internal routing and configuration chains; no observable behavioral or functional impact.

derive a log category - FATAL, ERROR, WARN, INFO, DEBUG, TRACE
for each ingested log records

first map severity_number directory to a category
fallback where severity_number is unset (0 / UNSPECIFIED),
scan the log body to find category based on case-insensitive substring match

defaults to UNSPECIFIED when neither rules yields a result
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 25, 2026

Walkthrough

Adds formatting-only trailing commas to HTTP routing builders and extends OpenTelemetry log processing with a new known field p_log_category, severity-to-category mapping, body-based fallback category derivation, and insertion of the resolved category into flattened log records.

Changes

Cohort / File(s) Summary
HTTP Handler Routing Formatting
src/handlers/http/modal/query_server.rs, src/handlers/http/modal/server.rs
Inserted trailing commas after chained .service(...) registrations in routing builder chains; purely formatting/bracketing changes.
OpenTelemetry Log Category Mapping
src/otel/logs.rs
Expanded OTEL_LOG_KNOWN_FIELD_LIST from 16→17 by adding p_log_category; added severity→category mapping, case-insensitive body-based fallback lookup, helper contains_ignore_ascii_case, and updated flatten_log_record to derive and insert p_log_category (preventing client override).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰
I hopped through commas, light and spry,
Gave logs new names beneath the sky,
From TRACE to FATAL, clear and neat,
A category snug in every beat,
I nibble bugs and leave the rest to you.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description lacks the required template structure, missing the 'Description' section, 'Fixes' reference, and testing/documentation checklist items specified in the repository template. Follow the repository's PR template: add a 'Description' section, include testing/documentation checklist items, and format according to the provided template structure.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'add category to OTel logs' clearly and concisely summarizes the main change—adding a category field to OpenTelemetry logs.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/otel/logs.rs`:
- Around line 178-181: Move the insertion of "p_log_category" so it occurs after
the call to insert_attributes to prevent client-provided attributes from
overwriting the server-derived value; locate the log_record_json.insert(...) for
"p_log_category" in src/otel/logs.rs and the insert_attributes(...) call in
src/otel/otel_utils.rs, remove the earlier insertion and re-insert
"p_log_category" immediately after insert_attributes returns (preserving the
same unwrap_or("UNSPECIFIED") behavior) so the server-set key wins.

ℹ️ Review info

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8859147 and 5edf0e6.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (3)
  • src/handlers/http/modal/query_server.rs
  • src/handlers/http/modal/server.rs
  • src/otel/logs.rs

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/otel/logs.rs (1)

102-114: Body fallback substring matching is prone to false positives.

Current partial-match logic can misclassify ("terror"ERROR, "debugger"DEBUG). Consider token-boundary matching for better category precision.

Also applies to: 163-174

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/otel/logs.rs` around lines 102 - 114, The substring match in
contains_ignore_ascii_case used by category_from_body is causing false
positives; change the matching to require token/word-boundary matches
(case-insensitive) instead of arbitrary substring windows: replace the
byte-windows approach with a word-boundary-aware check (e.g., regex with \b or
tokenize the body and compare whole tokens) when scanning LOG_CATEGORIES
patterns so patterns only match full words; apply the same change to the
duplicate logic referenced later in the file (the other occurrence that mirrors
contains_ignore_ascii_case/category_from_body).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@src/otel/logs.rs`:
- Around line 102-114: The substring match in contains_ignore_ascii_case used by
category_from_body is causing false positives; change the matching to require
token/word-boundary matches (case-insensitive) instead of arbitrary substring
windows: replace the byte-windows approach with a word-boundary-aware check
(e.g., regex with \b or tokenize the body and compare whole tokens) when
scanning LOG_CATEGORIES patterns so patterns only match full words; apply the
same change to the duplicate logic referenced later in the file (the other
occurrence that mirrors contains_ignore_ascii_case/category_from_body).

ℹ️ Review info

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5edf0e6 and c0a870a.

📒 Files selected for processing (1)
  • src/otel/logs.rs

@nitisht nitisht merged commit 0ccf8da into parseablehq:main Feb 25, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants