Skip to content

feat: byte-driven BLAKE3 hashing progress bar, TTY detection & --quiet flag#67

Merged
Rakshat28 merged 2 commits into
Rakshat28:mainfrom
Itzzavdheshh:stor
May 24, 2026
Merged

feat: byte-driven BLAKE3 hashing progress bar, TTY detection & --quiet flag#67
Rakshat28 merged 2 commits into
Rakshat28:mainfrom
Itzzavdheshh:stor

Conversation

@Itzzavdheshh

@Itzzavdheshh Itzzavdheshh commented May 24, 2026

Copy link
Copy Markdown
Contributor

📌 Related Issue

Fixes #46


📝 Description

🔹 What has been changed?

src/hasher.rs

  • Global thread-safe QUIET AtomicBool added with set_quiet() and is_quiet() helpers — worker threads read it directly, no state propagation needed
  • full_hash() updated to accept Option<&indicatif::ProgressBar> — bar advanced by exact bytes read per chunk:
if let Some(bar) = pb {
    bar.inc(read as u64);
}
  • sparse_hash() signature left unchanged — small files (≤12KB) that fall back to full hashing internally are excluded from progress bar to prevent double-counting

src/main.rs

  • Global -q / --quiet CLI flag added via Clap derive API — documented in custom GLOBAL FLAGS help template
  • std::io::IsTerminal check on stdout — non-TTY (piped) automatically activates quiet mode; zero ANSI pollution in pipes
  • byte_progress() helper builds bar with ProgressDrawTarget::stderr() + {bytes}/{total_bytes} ({bytes_per_sec}, {eta}) template
  • scan_pipeline accumulates total bytes dynamically as duplicate candidates are identified — bar length set in bytes not file count
  • All spinners/bars switch to ProgressBar::hidden() when quiet or non-TTY stdout detected
  • Unused file-count progress helper function removed — zero dead code warnings

tests/integration_tests.rs

  • test_quiet_suppresses_progress_output — verifies -q produces no stderr progress output while asserting command success
  • test_nontty_stdout_suppresses_progress_output — verifies piped execution auto-suppresses all progress rendering

🔹 Why are these changes needed?

  • The BLAKE3 full-hash pass — the most I/O-intensive part of any run — ran completely silently; on large directories the tool appeared frozen or hung
  • Progress bars wrote raw ANSI \r sequences to stdout — piped commands like bdstorage dedupe /path | grep something produced garbled, unparseable output
  • No way to suppress progress output in scripts or CI — --quiet flag was entirely absent

🛠️ Type of Change

  • 🐛 Bug Fix
  • ✨ New Feature
  • 🎨 UI/UX Improvement
  • 📖 Documentation Update
  • ⚡ Performance Improvement
  • 🔧 Refactoring
  • 💥 Breaking Change

🧪 Testing

cargo fmt --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test

✅ Tests Performed

  • Tested locally
  • Build runs successfully
  • No console errors
  • Responsive design checked
  • Existing functionality verified
  • cargo fmt --check — passes clean, zero violations
  • cargo test17/17 tests pass including 2 new integration tests
  • Large directory run — byte-driven progress bar with MB/s + ETA visible
  • Bar disappears cleanly on completion — no leftover ANSI artifacts
  • bdstorage dedupe /path | grep something — clean output, zero garbled \r sequences
  • -q flag — all progress output suppressed, command exits successfully
  • Double-counting verified absent — sparse_hash excluded from bar correctly

🌐 Browsers Tested

N/A — CLI tool


📷 Screenshots / Demo

1. Before — full-hash stage runs silently, tool appears hung
image


2. After — byte-driven progress bar with MB/s throughput and ETA
image


3. Bar disappears cleanly on completion
image


4. bdstorage dedupe /path | grep something — clean output, zero ANSI pollution
image


5. -q flag — zero progress output on stderr
image


6. cargo test — 17/17 tests passing including new integration tests
image


📋 Checklist

  • I have read the project's CONTRIBUTING guidelines
  • My code follows the project style guidelines
  • I have performed a self-review of my code
  • I have tested my changes locally
  • I have added/updated documentation where necessary
  • My changes do not introduce new warnings or errors
  • This PR is linked to an existing issue
  • Progress bar driven by bytes — shows real MB/s throughput and accurate ETA
  • All progress output bound to ProgressDrawTarget::stderr() — stdout untouched
  • Non-TTY stdout auto-activates quiet mode — pipes never polluted
  • -q / --quiet flag suppresses all progress output globally
  • sparse_hash excluded from bar — no double-counting on ≤12KB files
  • QUIET stored as AtomicBool — thread-safe, no complex state propagation
  • cargo fmt --check — pass
  • cargo test — 17/17 pass
  • Unused progress helper removed — zero dead code warnings
  • Existing dedupe and scan behaviour completely unchanged

💬 Additional Notes

  • Why AtomicBool for QUIET? Worker threads in the hashing pool need to check quiet state without locks or Arc passing — a global atomic is the idiomatic Rust pattern for this.
  • Why bytes not file count? File count gives misleading ETAs when files vary wildly in size. Bytes processed reflects actual I/O work done and maps directly to MB/s throughput.
  • Why stderr for progress? stdout must stay clean for piping. ProgressDrawTarget::stderr() is the standard indicatif pattern — progress never touches stdout.
  • Only 3 files changed — surgical, zero regressions on existing scan and dedupe flows.

🙌 Contribution Note

Hi @Rakshat28 👋

  • Silent hashing fixed — byte-driven bar with MB/s + ETA on the full BLAKE3 pass
  • Pipe pollution fixed — TTY check auto-suppresses all ANSI on non-TTY stdout
  • --quiet flag added — scripts and CI can fully silence progress output
  • Double-counting preventedsparse_hash correctly excluded from bar
  • 17/17 tests pass — 2 new integration tests cover quiet mode and non-TTY suppression
  • 3 files changed — zero regressions

Happy to address any feedback! 🚀


🏷️ Labels

#feature #cli #rust #progress-bar #indicatif #blake3 #tty #quiet-mode #integration-tests #NSoC26 #nsoc #GSSOC


Submitted as part of Open Source Contribution — NSoC (Nexus Spring of Code) & GSSoC

@Itzzavdheshh

Copy link
Copy Markdown
Contributor Author

Hi @Rakshat28 👋

  • Silent hashing fixed — byte-driven bar with MB/s + ETA on the full BLAKE3 pass
  • Pipe pollution fixed — TTY check auto-suppresses all ANSI on non-TTY stdout
  • --quiet flag added — scripts and CI can fully silence progress output
  • Double-counting preventedsparse_hash correctly excluded from bar
  • 17/17 tests pass — 2 new integration tests cover quiet mode and non-TTY suppression
  • 3 files changed — zero regressions

Happy to address any feedback! 🚀

@Rakshat28

Copy link
Copy Markdown
Owner

Please change README.md to reflect the addition of the --quiet flag @Itzzavdheshh

Repository owner deleted a comment from Itzzavdheshh May 24, 2026
Repository owner deleted a comment from Itzzavdheshh May 24, 2026
Repository owner deleted a comment from Itzzavdheshh May 24, 2026
@Itzzavdheshh

Copy link
Copy Markdown
Contributor Author

wow @Rakshat28 , thanku for your kind whatsapp message , i think we should not be working together .... i request you to close this PR unmerged and unassign me from your remaining issues ... its been great working with you , Thanku

Added documentation for global flags in README.
@Rakshat28 Rakshat28 merged commit 99b0d44 into Rakshat28:main May 24, 2026
2 checks passed
@Itzzavdheshh Itzzavdheshh deleted the stor branch May 24, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: progress reporting for full BLAKE3 hashing pass

3 participants