Skip to content

Emit metric when task count is capped by maxTasks#997

Merged
khandelwal-ayush merged 1 commit intolinkedin:masterfrom
khandelwal-ayush:inc-max-task
Feb 17, 2026
Merged

Emit metric when task count is capped by maxTasks#997
khandelwal-ayush merged 1 commit intolinkedin:masterfrom
khandelwal-ayush:inc-max-task

Conversation

@khandelwal-ayush
Copy link
Collaborator

Summary

  • Add a numTasksCappedByMaxTasks meter to StickyPartitionAssignmentStrategy that fires whenever the computed task count exceeds maxTasks and gets capped
  • Enables alerting on datastreams that are persistently hitting their maxTasks ceiling, which indicates they may need a higher limit or capacity adjustment

Details

When elastic task assignment computes a task count higher than the configured maxTasks, the strategy silently caps it and logs a warning. There was no metric to alert on this condition. This change adds a
per-datastream meter (StickyPartitionAssignmentStrategy..numTasksCappedByMaxTasks) so we can set up alerts for sustained capping, which signals that a datastream is under-provisioned.

Testing Done

Will deploy and test

@khandelwal-ayush khandelwal-ayush merged commit 2aa4f15 into linkedin:master Feb 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments