Skip to content

workday: fix infinite 401 retry loop in activity CEL program #20072

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
efd6 merged 1 commit into from
Jul 10, 2026
+26 −5
Merged

workday: fix infinite 401 retry loop in activity CEL program #20072

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/workday/_dev/deploy/docker/files/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ rules:
{
"access_token": "xxxx",
"token_type": "Bearer",
"expires_in": 3600,
"refresh_token": "refresh_token"
}
`}}
Expand Down
5 changes: 5 additions & 0 deletions packages/workday/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: '0.2.0'
changes:
- description: Fix token refresh after 401 by adding TTL-based expiry tracking.
type: bugfix
link: https://github.com/elastic/integrations/pull/20072
- version: '0.1.0'
changes:
- description: Add support for activity data stream.
Expand Down
23 changes: 19 additions & 4 deletions packages/workday/data_stream/activity/agent/stream/cel.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ program: |-
state.url.trim_right("/").as(base_url,
state.with(
(
(state.?cursor.access_token.orValue("") == "") ?
(!has(state.?cursor.access_token) ||
!has(state.?cursor.token_expiry) ||
timestamp(state.cursor.token_expiry) < now - duration("30s")) ?
post_request(
state.token_url,
"application/x-www-form-urlencoded",
Expand All @@ -58,6 +60,17 @@ program: |-
{
"ok": true,
"access_token": ao.access_token,
// Workday's "Enforce 60 Minute Access Token Expiry"
// is a server-side admin setting. When disabled, the
// token endpoint returns expires_in:-1 and tokens are
// "valid for several hours". Default to 60m in that
// case. This is conservative but well within the actual
// lifetime.
"token_expiry":
(!has(ao.expires_in) || int(ao.expires_in) < 0) ?
(now + duration("3540s")).format(time_layout.RFC3339)
:
(now + duration(string(int(ao.expires_in) - 60) + "s")).format(time_layout.RFC3339),
}
)
:
Expand All @@ -72,6 +85,7 @@ program: |-
{
"ok": true,
"access_token": state.cursor.access_token,
"token_expiry": state.cursor.token_expiry,
}
).as(tok,
!tok.ok ?
Expand Down Expand Up @@ -116,6 +130,7 @@ program: |-
"want_more": has_more,
"cursor": {
"access_token": tok.access_token,
"token_expiry": tok.token_expiry,
?"max_ingested_time": has_more ?
(
has(state.?cursor.max_ingested_time) ?
Expand Down Expand Up @@ -152,11 +167,11 @@ program: |-
"error": {
"code": string(resp.StatusCode),
"id": resp.Status,
"message": "GET " + base_url + "/activityLogging: token expired, retrying",
"message": "GET " + base_url + "/activityLogging: token expired",
},
},
"want_more": true,
"offset": state.offset,
"want_more": false,

@vera-review-bot vera-review-bot Bot Jul 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/workday/data_stream/activity/agent/stream/cel.yml.hbs:173

The 401 handler halts the loop but never invalidates the cached token, so a token the local TTL still considers valid is reused every interval until it lapses; expire the token in the cursor on 401 to force an immediate re-auth.

Details

The 401 branch returns {"events": {"error": ...}, "want_more": false, "offset": 0} to state.with(...). state.with merges into state, so because the returned object has no cursor key, the previous cursor (holding access_token and a still-future token_expiry) is preserved. On the next execution the refresh gate at the top of the program (!has(state.?cursor.access_token) || !has(state.?cursor.token_expiry) || timestamp(state.cursor.token_expiry) < now - duration("30s")) all evaluate false, so no refresh occurs and the same server-rejected token is sent again — producing another 401. This is precisely the case this branch exists to handle ("token expired"), yet nothing here forces a re-auth: recovery depends entirely on the tracked TTL elapsing rather than on the 401. Setting want_more: false correctly stops the tight in-run loop the PR targets, but a token that the server rejects before its tracked expiry will keep 401ing on each poll interval until token_expiry passes. The conservative TTL (60s margin subtracted; 3540s default) makes this unlikely in the normal Workday case, but the 401 path itself provides no recovery.

Recommendation:

On the 401 path, expire the cached token in the cursor so the next execution's refresh gate fires immediately, while preserving pagination progress (last_timestamp / max_ingested_time):

              : (resp.StatusCode == 401) ?
                {
                  "events": {
                    "error": {
                      "code": string(resp.StatusCode),
                      "id": resp.Status,
                      "message": "GET " + base_url + "/activityLogging: token expired",
                    },
                  },
                  "cursor": state.?cursor.orValue({}).with({
                    "token_expiry": (now - duration("1h")).format(time_layout.RFC3339),
                  }),
                  "want_more": false,
                  "offset": 0,
                }

This leaves access_token and the paging fields intact but backdates token_expiry, so the next run re-authenticates rather than replaying the rejected token.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@efd6 efd6 Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an intentional trade-off and the proposed solution is incorrect; it would not update the cursor since it an error object return. The trade-off is whether to attempt to resolve immediately silently (what the correct version of Vera's proposal would be), or to noisily fail a limited number of times (what the current code does). With the current code and default interval of 5m, there would be 12 error polls until the cursor's token expires; it is not a tight loop.

"offset": 0,
}
:
{
Expand Down
2 changes: 1 addition & 1 deletion packages/workday/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.3.2
name: workday
title: 'Workday'
version: 0.1.0
version: 0.2.0
description: 'Collect logs from Workday with Elastic Agent.'
type: integration
categories:
Expand Down
Loading