Skip to content

feat: support concurrent chunk uploads#314

Merged
TorstenDittmann merged 10 commits into
mainfrom
concurrent-chunk-uploads-1-9-x-minimal
May 21, 2026
Merged

feat: support concurrent chunk uploads#314
TorstenDittmann merged 10 commits into
mainfrom
concurrent-chunk-uploads-1-9-x-minimal

Conversation

@TorstenDittmann

Copy link
Copy Markdown
Contributor

This PR updates the SDK to support concurrent chunk uploads.

@greptile-apps

greptile-apps Bot commented May 21, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds concurrent chunk uploading to both the browser and native IO clients, replacing the sequential while loop with a worker-pool pattern (up to 8 concurrent workers) that shares a nextChunk counter — safe in Dart's single-threaded event loop because the check-and-increment happens before any await. Each worker in the IO client opens its own RandomAccessFile handle to avoid seek/read races, and headers/params are copied per-chunk to prevent shared-state mutations across concurrent calls.

  • Concurrent upload workers: both client_browser.dart and client_io.dart now build a flat chunks list and spawn min(8, chunks.length) async workers that pull from it; the first chunk is always uploaded serially to obtain the server-assigned uploadId before workers start.
  • Response handling: finalResponse is set only when isUploadComplete returns true inside a worker; for all other cases the code falls back to lastResponse; note that isUploadComplete is currently never invoked on the first chunk's response (see inline comments), meaning finalResponse can stay null even when the first-of-session chunk is also the last chunk of the file.
  • Version bump: SDK version updated from 24.1.1 to 24.2.0 across pubspec.yaml and both client headers.

Confidence Score: 4/5

The concurrent upload logic is sound for Dart's event-loop model, but the first-chunk response is never tested for upload completion, leaving a gap when the resumed first chunk is also the final chunk of the file.

The worker-pool design is correct and shared-state mutations are safe under Dart's cooperative scheduler. The first-chunk response omits the isUploadComplete check that every subsequent chunk gets inside uploadNext, meaning finalResponse stays null in that edge case and the caller silently receives the lastResponse fallback.

Both lib/src/client_browser.dart and lib/src/client_io.dart share the same first-chunk isUploadComplete omission and the lastResponse/finalResponse fallback behavior.

Important Files Changed

Filename Overview
lib/src/client_browser.dart Rewrites chunked upload to use up to 8 concurrent workers with a shared nextChunk counter; correctly copies headers/params per chunk and reports progress, but isUploadComplete is never checked on the first chunk's response
lib/src/client_io.dart Mirrors browser changes for native I/O; each worker opens its own RandomAccessFile to avoid shared-seek races; same isUploadComplete/lastResponse concerns as browser client apply
pubspec.yaml Version bump from 24.1.1 to 24.2.0 in both pubspec.yaml and the SDK headers in both clients

Reviews (4): Last reviewed commit: "Commit from GitHub Actions (Format and p..." | Re-trigger Greptile

Comment thread lib/src/client_io.dart Outdated
Comment thread lib/src/client_browser.dart
@TorstenDittmann TorstenDittmann merged commit 6bf15ec into main May 21, 2026
1 check passed
@TorstenDittmann TorstenDittmann deleted the concurrent-chunk-uploads-1-9-x-minimal branch May 21, 2026 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants