.gitignore (non repo root) gaps and overrides

[s-frantz]

Version

codebase-memory-mcp 0.8.1

Platform

Windows (x64)

Install channel

GitHub release archive / install.sh / install.ps1

Binary variant

standard

What happened, and what did you expect?

Hello, love the work you're putting into this project!

Have one more .gitignore -related suggestion to add alongside #489 & #499, plus a suggestion for #497... (theme being gitignore gaps & overrides) in case there's opportunity to batch all into one PR.

Specifically, I've noticed:

1. index_repository seems to miss .gitignore rules (both dir-level and interited from parents) when called on a folder not at repo root, e.g., a subpackage
2. .cbmignore might be a useful place to add in overrides as described in Support .ignore files that act as a local override layer on top of .gitignore. #497 but currently (when index_repository is called on a valid git repo root), is additive with .gitignore

Suggested solution for 1. would be to either warn the user, loudly, when non git repo is passed and gitignore is present (something to the effect of "NOT A GIT REPO -- .GITIGNORE WON'T BE RESPECTED"), or to take on the recursive journey up to find parent git repo (where other generally applied ignore rules might reside), or to do a combination and warn + respect any within-folder .gitignore files.

Suggested for 2 would be to take the same approach as git itself: give .cbmignore priority as last instruction, so any conflicts (e.g., .gitignore shows dir, but .cbmignore shows !dir / !dir/* -> leads to dir being included in index).

MfG
Silas

Reproduction

Repro 1 — .gitignore skipped when indexing a non-root subfolder

T=$(mktemp -d); cd "$T"
git init -q                                                   # repo root = $T
mkdir pkg
echo 'def secret(): return "SECRET_TOKEN_111"' > pkg/secret.py
echo 'def ok(): return 1'                       > pkg/keep.py
echo 'secret.py' > pkg/.gitignore                             # committed, at the subpackage's own root
git add -A && git commit -qm init

# git itself agrees secret.py is ignored, even addressed from inside pkg:
git -C "$T/pkg" check-ignore secret.py
#   secret.py        ← git says: ignored

# (A) index the SUBPACKAGE (not the repo root):
OUT=$(codebase-memory-mcp cli index_repository "{\"repo_path\":\"$T/pkg\"}")
echo "$OUT"                                                   # note: "excluded" lists no .gitignore'd entry
PKG=$(printf '%s' "$OUT" | sed -n 's/.*"project":"\([^"]*\)".*/\1/p')
codebase-memory-mcp cli search_code "{\"pattern\":\"SECRET_TOKEN_111\",\"project\":\"$PKG\"}"
#   EXPECTED (bug): a hit — secret.py was indexed despite pkg/.gitignore

# (B) contrast: index the actual repo ROOT — same files, same .gitignore:
OUT2=$(codebase-memory-mcp cli index_repository "{\"repo_path\":\"$T\"}")
ROOT=$(printf '%s' "$OUT2" | sed -n 's/.*"project":"\([^"]*\)".*/\1/p')
codebase-memory-mcp cli search_code "{\"pattern\":\"SECRET_TOKEN_111\",\"project\":\"$ROOT\"}"
#   EXPECTED (correct): no hit — nested pkg/.gitignore is honored when indexing the root

# cleanup
codebase-memory-mcp cli delete_project "{\"project\":\"$PKG\"}"
codebase-memory-mcp cli delete_project "{\"project\":\"$ROOT\"}"
rm -rf "$T"

Repro 2 — .cbmignore is additive; ! can't override .gitignore (on a valid repo root)

T=$(mktemp -d); cd "$T"
git init -q
mkdir build extra
echo 'def gen():    return "OVERRIDE_MARKER_222"' > build/gen.py
echo 'def x():      return "CONTROL_MARKER_333"'  > extra/x.py
echo 'def normal(): return 1'                     > main.py
echo 'build/' > .gitignore                                    # gitignore excludes build/
printf '!build/\nextra/\n' > .cbmignore                       # try to re-include build/, AND plainly exclude extra/
git add -A && git commit -qm init

OUT=$(codebase-memory-mcp cli index_repository "{\"repo_path\":\"$T\"}")
echo "$OUT"
PROJ=$(printf '%s' "$OUT" | sed -n 's/.*"project":"\([^"]*\)".*/\1/p')

codebase-memory-mcp cli search_code "{\"pattern\":\"CONTROL_MARKER_333\",\"project\":\"$PROJ\"}"
#   EXPECTED: no hit → proves .cbmignore IS loaded and applied (its `extra/` exclude works)

codebase-memory-mcp cli search_code "{\"pattern\":\"OVERRIDE_MARKER_222\",\"project\":\"$PROJ\"}"
#   EXPECTED (current behavior): no hit → `!build/` did NOT re-include; .gitignore wins, cbmignore is additive

# cleanup
codebase-memory-mcp cli delete_project "{\"project\":\"$PROJ\"}"
rm -rf "$T"

Logs

Project scale (if relevant)

No response

Confirmations

• I searched existing issues and this is not a duplicate.
• My reproduction uses shareable code (a dummy snippet or a public OSS repository), not proprietary code.

[s-frantz]

aha, I see you already suggested the same fix for 2.^ in comments here

[ecosuper2025]

+1 from a ~5k-file polyglot monorepo on Windows (Capacitor-based mobile-app workspace, codebase-memory-mcp 0.8.1).

My data extends Repro 1-B's "expected (correct)" claim: even when index_repository is called on the actual repo root, the nested .gitignore is NOT honored, AND adding the same pattern to the root .gitignore doesn't help either.

Setup

• Repo e:/mercadoya, 5,003 files, called index_repository {"repo_path":"E:/mercadoya"} from the root.
• apps/merchant-web/android/app/src/main/assets/public/ is a ~63 MB Capacitor cap sync artifact mirror (vite dist/ copied for APK packaging), excluded by apps/merchant-web/android/.gitignore:99 (app/src/main/assets/public).
• git check-ignore -v apps/merchant-web/android/app/src/main/assets/public/index.html → confirms git itself ignores it via the nested .gitignore.

Result — vendor bundles inside that nested-ignored dir dominate the graph

• total_nodes=93,315, total_edges=477,188.
• Top 10 hotspots: 9 of 10 are minified vendor chunks — vendor-charts-CLZY7-BD.Yn.concat fan_in=1800, vendor-antd-CZzxTBtm.Dl.map fan_in=1637, etc. The top real-code hotspot (backend/core/services/llm_provider_chain.LLMChain.call fan_in=1346) is buried at Add ruby support #4.
• 18,985 SIMILAR_TO edges, the majority from minified-chunk pseudo-duplicates.

Reproducing Repro 1-B mode (root call) — root .gitignore glob also ignored

Added the matching pattern to root .gitignore:

apps/*/android/app/src/main/assets/public/

Re-indexed → total_nodes stayed at 93,315 → 93,315. search_graph {"name_pattern":"vendor-charts"} still returns 4 hits in that path. The excluded.dirs field in the result still shows only the hard-coded list (.claude .git ... rider-web/android/app/build).

So two things stack on top of each other:

1. Nested .gitignore not honored even from a root call (extends Repro 1-B which assumed it would be).
2. Root .gitignore glob patterns (apps/*/...) also appear not honored — only the hard-coded ALWAYS_SKIP_DIRS ever shows up in excluded.dirs (which is what Please make ALWAYS_SKIP_DIRS configurable by user #500 asks to expose).

Net effect: for monorepos with framework-managed sync artifacts (Capacitor / Tauri / Expo / Nx) there's currently no in-index way to keep them out, even though git itself ignores them. The only workaround is file_pattern exclusion at query time, which doesn't reduce index size, RAM, or graph noise.

Happy to test patches against this repo if useful. Thanks for the great project!

[DeusData]

This issue now has a first-class, durable reproduction in the cumulative bug-reproduction suite landing in #667 — a test that pins the exact failing behaviour, so it's tracked and guarded against silent regression. Where #667 also lands the root-cause fix, that's called out in the PR; otherwise the reproduction makes the eventual fix straightforward and verifiable.

The reproduction goes live once #667 merges. (Posted from the release branch's QA pass.)