Add reliable message queues

[zzstoatzz]

this PR adds a small Redis Streams queue primitive for runtimes that need Docket reliability without delegating execution or results to Docket.

This is motivated by PrefectHQ/prefect#21218: Prefect owns task execution, state, and result persistence, but its API replicas need a shared, crash-recoverable delivery layer.

API and semantics

• docket.queue(name) creates a keyed queue.
• put() publishes opaque bytes with queue-wide idempotency and optional topic backpressure.
• subscribe() provides competing consumers across FIFO topics, with priority among ready messages.
• acknowledge() durably deletes accepted work.
• release() atomically moves work to another topic for immediate retry.
• Unacknowledged work is reclaimed after a visibility timeout; active subscriptions renew visibility.
• Optional acknowledgement tombstones support external repair loops without coupling Docket to their state or result model.

Delivery is intentionally at least once. The downstream runtime remains responsible for idempotent execution and results.

Implementation

The queue uses Redis Streams consumer groups, XAUTOCLAIM, and small atomic Lua operations for publish, acknowledge, and release. Redis keys, clients, group recovery, visibility renewal, and deduplication storage remain private Docket details.

One subscription uses a constant two background tasks and a single multi-stream reader regardless of topic count. Every claimed message—including Redis-side prefetch—is visibility-renewed, and graceful shutdown releases internal buffers immediately. Acknowledged consumers can advance while the durable Redis acknowledgement completes; if that write fails, the original stream entry remains and is redelivered.

The implementation reuses Docket existing standalone, cluster, Sentinel, ACL, and in-memory connection paths.

Validation

• 25 queue tests cover FIFO and priority routing, idempotency, bounded publish/release, same-topic release, competing consumers, multi-topic scaling, abandoned-message reclamation, prefetched-message renewal, graceful buffer release, acknowledgement failure/redelivery, Redis reconnects, consumer-group recreation, memory-backend compatibility, payload validation, and lifecycle behavior.
• The queue suite passes against the in-memory backend and Redis 6.2, the minimum version supporting XAUTOCLAIM.
• The new queue modules have 100% statement and branch coverage.
• uv run prek run --all-files passes, including Ruff, file-size limits, source/test Pyright, and 100% public type completeness.
• A VCS-less source archive builds successfully using the 0.0.0 fallback version, allowing immutable HTTPS pins in slim images without Git.
• Full suite functional result: 983 passed, 7 skipped. On this macOS/Python 3.14 run, the repository-wide 100% gate reports 99.90% from pre-existing platform/concurrency branches in _lua.py, _redis.py, _concurrency.py, and worker tests; no queue line or branch is missing.
• The exact Python 3.10 + Valkey 8.1 CI seed passes locally (877 passed, 7 skipped in 83.66s). That GitHub leg alone reaches 89% before its four-minute job cap; the other 53 checks pass.

Compatibility

Existing task scheduling and result APIs are unchanged. Queue use requires Redis 6.2 or newer; Docket task execution continues to require only Redis Streams support.

[codecov-commenter]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
988	1	987	3

View the top 1 failed test(s) by shortest run time

tests/test_queue.py::tests.test_queue

Stack Traces | 0s run time

tests/test_queue.py:366: in <module>
    async def _redis_connection(redis: AsyncMock) -> AsyncGenerator[AsyncMock]:
                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^
....../usr/lib/python3.12/typing.py:398: in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
....../usr/lib/python3.12/typing.py:1455: in __getitem__
    _check_generic(self, params, self._nparams)
.venv/lib/python3.12/site-packages/typing_extensions.py:3274: in _check_generic
    raise TypeError(f"Too {'many' if alen > elen else 'few'} arguments"
E   TypeError: Too few arguments for typing.AsyncGenerator; actual 1, expected 2

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[read-the-docs-community]

Documentation build overview

📚 docket | 🛠️ Build #33841664 | 📁 Comparing 6d05b3a against latest (56a16f7)

  🔍 Preview build  

11 files changed · + 1 added · ± 10 modified

+ Added

• queues/index.html

± Modified

• 404.html
• index.html
• api-reference/index.html
• dependency-injection/index.html
• getting-started/index.html
• observability/index.html
• production/index.html
• task-behaviors/index.html
• task-patterns/index.html
• testing/index.html