Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,7 @@
## 2024-07-30 - Avoid chained string replace when checking character sets
**Learning:** Using chained `.replace(a, "").replace(b, "")` to check if a string consists entirely of specific characters requires intermediate string allocations for every call. In benchmarks, using `.strip("ab")` is ~30% faster and avoids multiple allocations in the hot path.
**Action:** When checking if a string is solely composed of specific characters, use `.strip(chars)` instead of chained `.replace()` calls to improve performance.

## 2024-08-01 - Performance Optimization: Increase Async File Read Chunk Size
**Learning:** In FastAPI, reading large file uploads with a small chunk size (e.g., 8KB) via `await file.read(8192)` causes excessive asynchronous event loop iterations and thread-dispatch overhead due to `SpooledTemporaryFile` mechanics.
**Action:** Increasing the chunk size (e.g., to 1MB: `await file.read(1024 * 1024)`) significantly reduces overhead and improves I/O performance for large uploads without sacrificing readability.
4 changes: 3 additions & 1 deletion src/newsdom_api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,9 @@ async def parse(
tmp.write(header)

bytes_read = len(header)
while chunk := await file.read(8192):
# ⚡ Bolt: Increase chunk size to 1MB to reduce async event loop iterations
# and thread-dispatch overhead for large PDF uploads.
while chunk := await file.read(1024 * 1024):
bytes_read += len(chunk)
if bytes_read > MAX_PARSE_UPLOAD_BYTES:
LOGGER.warning(
Expand Down
Loading