Skip to content

Add timestamp instrumentation for latency measurement#2620

Open
oteffahi wants to merge 32 commits into
eclipse-zenoh:mainfrom
ZettaScaleLabs:feat/routing-timestamps
Open

Add timestamp instrumentation for latency measurement#2620
oteffahi wants to merge 32 commits into
eclipse-zenoh:mainfrom
ZettaScaleLabs:feat/routing-timestamps

Conversation

@oteffahi

@oteffahi oteffahi commented May 26, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds the ability to record timestamps on messages along the Zenoh network path, notably to measure latency and identify bottlenecks in Zenoh routing.

What does this PR do?

Adds TimestampStack extension to application messages (Push, Query, Response) and expose API to configure instrumentation and access collected timestamps.

Sister PR for Python bindings: eclipse-zenoh/zenoh-python#736


🏷️ Label-Based Checklist

Based on the labels applied to this PR, please complete these additional requirements:

Labels: new feature

🆕 New Feature Requirements

Since this PR adds a new feature:

  • Feature scope documented - Clear description of what the feature does and why it's needed
  • Minimum necessary code - Implementation is as simple as possible, doesn't overcomplicate the system
  • New APIs well-designed - Public APIs are intuitive, consistent with existing APIs
  • Comprehensive tests - All functionality is tested (happy path + edge cases + error cases)
  • Examples provided - Usage examples in code comments or separate example files
  • Documentation added - New docs explaining the feature, its use cases, and API
  • Feature flag considered - Can the feature be enabled/disabled for gradual rollout?
  • Performance impact assessed - Memory, CPU, storage implications measured
  • Integration tested - Feature works with existing features

Consider: Can this feature be split into smaller, incremental PRs?

Instructions:

  1. Check off items as you complete them (change - [ ] to - [x])
  2. The PR checklist CI will verify these are completed

This checklist updates automatically when labels change, but preserves your checked boxes.

@oteffahi oteffahi marked this pull request as draft May 26, 2026 08:03
@oteffahi oteffahi added the new feature Something new is needed label May 26, 2026
@codecov

codecov Bot commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.86175% with 66 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.17%. Comparing base (e6a1d4e) to head (28a17a4).
⚠️ Report is 7 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
zenoh/src/api/timestamp_stack.rs 87.36% 23 Missing ⚠️
zenoh/src/api/builders/sample.rs 27.27% 16 Missing ⚠️
zenoh-ext/src/advanced_publisher.rs 0.00% 11 Missing ⚠️
commons/zenoh-codec/src/network/timestamp_stack.rs 88.33% 7 Missing ⚠️
zenoh/src/api/builders/session.rs 70.00% 3 Missing ⚠️
zenoh/src/api/query.rs 71.42% 2 Missing ⚠️
zenoh/src/net/runtime/adminspace.rs 85.71% 2 Missing ⚠️
zenoh/src/net/routing/dispatcher/pubsub.rs 95.00% 1 Missing ⚠️
zenoh/src/net/runtime/mod.rs 97.22% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2620      +/-   ##
==========================================
+ Coverage   74.11%   74.17%   +0.06%     
==========================================
  Files         400      403       +3     
  Lines       61044    61666     +622     
==========================================
+ Hits        45240    45743     +503     
- Misses      15804    15923     +119     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@YuanYuYuan YuanYuYuan force-pushed the feat/routing-timestamps branch from 2ebe61e to fff285c Compare May 28, 2026 17:44
@oteffahi oteffahi marked this pull request as ready for review June 15, 2026 17:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces timestamp instrumentation for Zenoh messages to measure latency along the network path by carrying a TimestampStack extension through Push/Query/Response flows, with configurable interception points (Send/Route/Receive) and optional custom timestamp generation via a session callback.

Changes:

  • Add a new timestamp_stack protocol extension (and codec support) for Push/Request/Response network messages.
  • Add unstable public API to configure instrumentation per message, access the collected stack on receive, and register a custom timestamp callback on session open.
  • Integrate interception pushes in routing and runtime paths (send/route/receive), and add extensive integration tests.

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
zenoh/tests/timestamp_instrumentation.rs New integration test suite covering pub/sub, routed, query/reply, delete, and custom callback behaviors.
zenoh/src/tests/query_reply.rs Adjust test struct initialization to include new unstable fields.
zenoh/src/net/tests/tables.rs Update network tests to initialize ext_ts_stack: None.
zenoh/src/net/tests/regions/mod.rs Update mock messages to include ext_ts_stack: None.
zenoh/src/net/runtime/mod.rs Add runtime timestamp callback plumbing and lazy UHLC initialization for instrumentation.
zenoh/src/net/runtime/adminspace.rs Push RECEIVE interceptions for adminspace-delivered Push/Request and propagate runtime into QueryInner.
zenoh/src/net/routing/dispatcher/queries.rs Propagate/cloned ext_ts_stack across routed requests and push ROUTE interceptions on forwarding.
zenoh/src/net/routing/dispatcher/pubsub.rs Propagate/cloned ext_ts_stack across routed pushes and push ROUTE interceptions on forwarding.
zenoh/src/lib.rs Expose new unstable timestamp_stack module and internal builder trait export.
zenoh/src/api/timestamp_stack.rs New unstable API types (stack, records, builder, callback context) + internal push helper + protocol conversions.
zenoh/src/api/session.rs Plumb timestamp callback into runtime; add timestamp stack propagation and RECEIVE interception before subscriber callbacks; support instrumentation on put/get.
zenoh/src/api/sample.rs Store/return timestamp stack on received samples; extend callback parameter tuple on unstable.
zenoh/src/api/queryable.rs Store/return timestamp stack on queries; propagate stack into replies and push SEND interception.
zenoh/src/api/query.rs Attach timestamp stack to ReplyError on unstable.
zenoh/src/api/querier.rs Initialize querier get builder with new instrumentation option.
zenoh/src/api/publisher.rs Initialize publication builders with new instrumentation option; plumb into resolve_put.
zenoh/src/api/mod.rs Register new timestamp_stack API module behind unstable.
zenoh/src/api/builders/session.rs Add OpenBuilder::with_timestamp_callback (unstable).
zenoh/src/api/builders/sample.rs Add TimestampInstrumentationBuilderTrait and store instrumentation in Sample (unstable).
zenoh/src/api/builders/reply.rs Propagate query stack into reply error responses and push SEND interception.
zenoh/src/api/builders/query.rs Add per-get builder support for timestamp instrumentation (unstable).
zenoh/src/api/builders/querier.rs Add per-querier-get builder support for timestamp instrumentation (unstable).
zenoh/src/api/builders/publisher.rs Add per-publication builder support for timestamp instrumentation (unstable).
zenoh/src/api/admin.rs Adjust internal publish calls to account for new resolve_put parameter.
zenoh-ext/src/advanced_publisher.rs Forward timestamp instrumentation builder trait through advanced publisher wrapper.
commons/zenoh-protocol/src/network/timestamp_stack.rs New protocol definition for timestamp stack extension types and flags.
commons/zenoh-protocol/src/network/response.rs Add ext_ts_stack extension to Response and rand generation.
commons/zenoh-protocol/src/network/request.rs Add ext_ts_stack extension to Request and rand generation.
commons/zenoh-protocol/src/network/push.rs Add ext_ts_stack extension to Push and rand generation/defaults.
commons/zenoh-protocol/src/network/mod.rs Export new timestamp_stack module.
commons/zenoh-codec/src/network/timestamp_stack.rs New codec for timestamp stack extension encoding/decoding.
commons/zenoh-codec/src/network/response.rs Encode/decode Response ext_ts_stack.
commons/zenoh-codec/src/network/request.rs Encode/decode Request ext_ts_stack.
commons/zenoh-codec/src/network/push.rs Encode/decode Push ext_ts_stack.
commons/zenoh-codec/src/network/mod.rs Register timestamp stack codec module.
commons/zenoh-codec/fuzz/src/samples.rs Update fuzz samples to include ext_ts_stack: None.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread zenoh/src/api/timestamp_stack.rs
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Comment thread zenoh/src/net/runtime/mod.rs Outdated
Comment thread zenoh/src/net/runtime/mod.rs Outdated
Comment thread zenoh/src/api/session.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 36 out of 36 changed files in this pull request and generated 6 comments.

Comment thread zenoh/src/api/timestamp_stack.rs
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Comment thread zenoh/src/net/runtime/mod.rs Outdated
Comment thread zenoh/src/net/runtime/mod.rs Outdated
Comment thread zenoh/tests/timestamp_instrumentation.rs
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Comment thread zenoh/src/api/timestamp_stack.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new feature Something new is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants