feat: consolidate 19 community PRs, fix 22 bugs, upgrade Go 1.24#859
Open
terryrankine wants to merge 57 commits intopeak:masterfrom
Open
feat: consolidate 19 community PRs, fix 22 bugs, upgrade Go 1.24#859terryrankine wants to merge 57 commits intopeak:masterfrom
terryrankine wants to merge 57 commits intopeak:masterfrom
Conversation
- Add missing {{.HelpName}} before --content-type in cp help (peak#857)
- Fix typos: "send"→"sent", "anohter"→"another", "are adjustable"→"is adjustable" (peak#828)
- Update homebrew install command to `brew install s5cmd` (peak#783)
- Reword sync --include example text for clarity (peak#774)
- Add --raw option note in commands file section (peak#781)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bitnami/minio has been moved to bitnamilegacy/minio (peak#840) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove Profile from the credential-file-only branch so --profile alone uses the SDK's shared config resolution (SSO, assume-role, etc.) instead of being forced through NewSharedCredentials. Pass Profile into session.Options so the SDK picks up the correct profile. (peak#847) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the IsRegular() check after the exclude/include filter so that non-regular files matching an exclude pattern are silently skipped instead of producing spurious error messages. (peak#776) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add nil check for srcObj after statObject returns successfully but with no object found, preventing a segfault when comparing source and destination metadata. (peak#838) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add IsTencentEndpoint() to detect .myqcloud.com endpoints and enable virtual-host-style bucket resolution for Tencent COS, matching existing Google Cloud Storage support. (peak#809) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace fmt.Sprintf("%q") with shellquote.Join() for URL arguments in
generated commands. Go's %q produces Go-style escaping (backslash
sequences) that breaks shell parsing of object keys containing special
characters like brackets or ampersands. shellquote.Join() produces
proper POSIX shell quoting. (peak#761)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the progress bar is a NoOp (disabled), skip the extra Stat call used to populate object size for progress tracking. This avoids an unnecessary API call per object when --show-progress is not set. (peak#793) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add RequestError and SerializationError to shouldStopSync, preventing sync from continuing after network/serialization failures - Add context cancellation checks in planRun goroutines so they terminate promptly on SIGINT/cancel instead of blocking on channels (peak#698) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Separate ExpiredToken/ExpiredTokenException (retryable after clearing the session cache) from InvalidToken (non-retryable). Long-running operations that outlive temporary credentials will now automatically refresh and retry instead of failing. (peak#683) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a server-side CopyObject fails because the source exceeds 5 GiB, fall back to multipart copy (CreateMultipartUpload + UploadPartCopy + CompleteMultipartUpload). Aborts are performed with a background context to ensure cleanup even after cancellation. (peak#856) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes EKS Pod Identity credential resolution and picks up other SDK improvements and bug fixes. (peak#769) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add codespell CI workflow and .codespellrc config. Fix typos in source and test comments: interfers→interferes, seperator→separator, succesfull→successful, charactes→characters, hiearchy→hierarchy, overriden→overridden. (peak#701) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
opts.region was incorrectly assigned opts.accessKeyID instead of opts.region, causing tests with custom regions to silently use the access key ID as the region value. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
checkNumberOfArguments reported min instead of max when too many arguments were given, producing confusing error messages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IncrementCompletedObjects and IncrementTotalObjects used atomic writes but non-atomic reads of the counters in fmt.Sprintf, creating a data race under concurrent worker access. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Statistics() iterated over internal maps without holding their mutexes, racing with concurrent Collect() calls from worker goroutines. This could cause a panic from concurrent map iteration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add nil checks for metadata string pointers before dereferencing in Stat() and retryOnNoSuchUpload(). Initialize metadata map before writing retry ID in Copy() and Upload(). Close pipe and propagate write errors in Select(). Preserve source metadata in multipart copy. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add missing defer cancel() after context.WithCancel() to prevent context leak. Replace os.Exit(1) on 'too many open files' with context cancellation for graceful shutdown. Replace goto with idiomatic loop-break pattern. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace os.Exit(1) on 'too many open files' with context cancellation
for graceful shutdown. Replace panic("unexpected src-dst pair") with
error return.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace zero-value &os.File{} with os.NewFile(0, os.DevNull) in
Create() and CreateTemp() dry-run paths. The zero-value File would
panic on any method call.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- actions/checkout v2 → v4 - actions/setup-go v2 → v5 - docker/setup-qemu-action v1 → v3 - docker/setup-buildx-action v1 → v3 - docker/login-action v1 → v3 - docker/build-push-action v2 → v6 - goreleaser/goreleaser-action v2 → v6 (use latest version, --clean) - Drop Go 1.20 from CI matrix (EOL since Feb 2024) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
If Init() was never called, Run() would panic with a nil pointer dereference. Now panics with an explicit message instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace %v with %w in fmt.Errorf calls for endpoint parsing and region fetching errors, preserving the error chain for callers using errors.Is/errors.As. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document all changes since v2.3.0 including new features (multipart copy, Tencent COS), AWS SDK bump, and all bug fixes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sync command was ignoring --source-region and --destination-region flags because it used the same storageOpts for both clients. Now creates separate option copies with region overrides, matching the cp command's existing pattern. (peak#858) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Configure explicit buffer pools for the S3 download (64 KiB) and upload (512 KiB) managers to reduce memory allocations and improve throughput, especially on network block devices. (peak#843) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --start-after flag to the ls command to support listing S3 objects starting after a specific key. Useful for pagination and resuming listings from a specific point. Passes StartAfter to the S3 ListObjectsV2 API. (peak#850) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add --log-progress (-lp) flag to the cp command that outputs progress in a pipe-friendly format to stderr. Uses newlines instead of carriage returns, no ANSI codes, updates every 2 seconds. Shows percentage, bytes transferred, speed, ETA, and file count. (peak#853) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add riscv64 to goreleaser build targets and Docker multi-platform builds. (peak#842) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
maybeUpdate() reads and writes lastUpdate from multiple worker goroutines without synchronization. Add sync.Mutex to prevent the data race. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use sync.Once to prevent double-close panic on the output channel if Close() is called more than once. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the NoOp type assertion out of the per-object loop to avoid repeated interface checks from concurrent goroutines. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The error handler goroutines for extsort could leak if the parent context was cancelled. Drain errors synchronously after the sorted output channel is consumed, eliminating the leak. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Catalogues all 136 open issues on peak/s5cmd, cross-referenced with our branch. 18 fixed, 25 improved, 73 not fixed, 20 questions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ls --humanize showed raw numbers like "123" for files under 1KB. Now shows "123B" for consistency with the K/M/G/T suffixes. (peak#817) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Objects matching --exclude patterns or not matching --include patterns are no longer deleted from the destination during sync --delete. This matches AWS CLI behavior where excluded paths are preserved. (peak#815) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- TestHumanizeBytes: verify B/K/M/G/T suffixes including byte range - TestCheckNumberOfArguments: verify max (not min) in error message - TestWithStartAfter: verify URL option sets StartAfter field - TestRunPanicsWithoutInit: verify parallel.Run panics without Init - TestInitWithLogFile: verify log output redirects to file - TestCloseMultipleTimes: verify log.Close is idempotent - TestStatisticsCollect: verify stat counting with success/error - TestStatisticsDisabled: verify empty stats when not initialized - TestIsVirtualHostStyleWithAddressingStyle: verify --addressing-style flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Table-driven tests verifying that AccessDenied, NoSuchBucket, RequestError, and SerializationError all stop sync, while other errors respect the exitOnError flag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for the pure formatting functions used by LogProgressBar, covering boundary values (B/kB/MB/GB) and duration formatting (seconds, minutes, hours). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Verifies that Create() and CreateTemp() in dry-run mode return valid file handles (os.DevNull) without creating actual files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When using --if-source-newer, srcObj.ModTime or dstObj.ModTime could be nil (e.g. some storage backends don't return modification times), causing a nil pointer dereference panic. Now proceeds with copy when either timestamp is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
terryrankine
requested review from
igungor and
seruman
and removed request for
a team
The vendored staticcheck panicked on Go 1.24's *types.Alias. Upgrade staticcheck and unparam to versions compatible with Go 1.24+. Also fix nil ModTime deref in shouldOverride for --if-source-newer. Dependency changes: - honnef.co/go/tools v0.4.7 → v0.6.0 - mvdan.cc/unparam → latest - golang.org/x/tools v0.21.0 → v0.29.0 - golang.org/x/mod v0.17.0 → v0.25.0 - golang.org/x/sys v0.20.0 → v0.29.0 - BurntSushi/toml v1.2.1 → v1.4.1 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Disable SA1019 (deprecated API usage) in staticcheck since we intentionally use the legacy extsort SortType API for compatibility - Fix TestSyncExitOnErrorS3BucketToS3BucketThatDoesNotExist to match "status code: 404" instead of "NoSuchBucket" (gofakes3 returns "NotFound" on Windows) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The 'to' URL parameter was unused since destination bucket/key come from originalInput. Caught by unparam. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
terryrankine
changed the title
Replace deprecated extsort.New/SortType/SortTypeSorter with the type-safe extsort.Generic[storage.Object] API: - FromBytes returns (Object, error) instead of SortType - ToBytes returns ([]byte, error) instead of []byte - Compare replaces Less with int return (-1/0/1) - Remove all type assertions in sync.go - url.FromBytes returns *URL directly - Revert SA1019 suppression in Makefile (no longer needed) - Add TestObjectSerializeRoundTrip and TestObjectCompare Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix struct field alignment in LogProgressBar (gofmt) - Remove extra blank lines left from extsort import removal (gofmt) - Rename test string 'abd' to 'xyz' to avoid codespell false positive Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skip GCS e2e test when GCS secrets are not configured (forks). Goreleaser job proceeds regardless using if: always() && !failure(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Job-level if cannot access secrets context. Move the conditional to step level using env var check instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add version: 2 header (required by goreleaser v2+) - Replace deprecated archives.replacements with name_template - Replace deprecated brews.tap with brews.repository Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- snapshot.name_template -> snapshot.version_template - archives.format_overrides.format -> formats (list) - Remove arm (v6) target: broken on windows, not useful - Add explicit ignore for darwin/riscv64, windows/ppc64le, windows/riscv64 (unsupported platforms) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Consolidates 19 open community PRs and fixes 22 open issues in a single branch. Upgrades Go to 1.24, AWS SDK to v1.55.5, and all CI tooling.
Community PRs Applied
--log-progressflag for non-TTY progress output--start-afterflag forlspagination--profileflag for AWS credential loading (SSO, assume-role)--addressing-styleflag for virtual host style endpoints--log-fileflag for logging to fileIssues Fixed
Bugs Found and Fixed During Code Review
Refactoring
Infrastructure Upgrades
Tests Added
18 new test functions across 7 new test files. Packages that previously had no tests now have coverage:
log,log/stat,parallel.Security
govulncheckclean: 0 called vulnerabilities in vendored dependencies.Test plan
Full issue catalogue:
docs/issue-tracker.mdGenerated with Claude Code