Skip to content

flex init leaks ~2 fds per chunk and dies with EMFILE on large first-time indexes #5

Description

@alecmarcus

Summary

flex init --module claude-code leaks file descriptors while indexing and exhausts the macOS per-process limit, killing the install partway through.

What happened

  Indexing 9,512 sessions

  Enrichment error: [Errno 24] Too many open files: '.../flex/modules/claude_code/enrichment.py'
  Scanning sessions    ✓ 9,512 sessions scanned
  Building vectors     ✓ 120,855 chunks (embedding skipped)
  Building graph       ✓ done

  1,233 sessions · 120,855 chunks

  Install failed: [Errno 24] Too many open files
Traceback (most recent call last):
  File "flex/cli.py", line 917, in cmd_init
  File "flex/modules/claude_code/install.py", line 263, in run
    _install_systemd() or _install_launchd()
  File "flex/cli.py", line 434, in _install_launchd
    probe = subprocess.run(["launchctl", "print", f"user/{uid}/{label}"], ...)
  ...
  File ".../subprocess.py", line 1885, in _execute_child
    errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files

By the end the process could not open anything: not enrichment.py (so enrichment was skipped), not the ONNX model (so embedding was skipped — note "embedding skipped" above), and finally not even a pipe for subprocess.run, which is where it died.

This is not a low ulimit

ulimit -Sn                 1048576
kern.maxfilesperproc       245760
kern.maxfiles              491520

Measured ceiling on this machine, opening /dev/null in a loop under the same interpreter:

python sees RLIMIT_NOFILE: 1048576 ...
EMFILE after opening 245757 fds

The soft limit is already above kern.maxfilesperproc, which is the real cap — so raising ulimit -n is not available as a workaround. flex genuinely held ~245k descriptors.

Scale

9,512 sessions / 120,855 chunks against a 245,757 ceiling works out to roughly two descriptors leaked per chunk (120,855 × 2 = 241,710, plus process baseline ≈ the ceiling). That ratio is inferred from the arithmetic — I did not isolate the leak site, and the shape is consistent with a per-chunk SQLite connection whose db/wal handles are never closed.

Notes

Re-running after the crash completed successfully — the expensive scan was already done, so far fewer chunks were freshly processed. During that run the process sat flat at 437 fds. So the leak scales with newly processed chunks, and large first-time indexes are the exposure.

Users with big session histories will hit this on their very first install, which is also the moment they have the least context to diagnose it.

Environment

getflex 0.52.0 · macOS (Darwin 25.5.0, arm64) · Python 3.14.3 · module claude-code · 9,512 sessions

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions