Skip to content

Reuse entity graph during bench runs#21

Closed
Iron-Ham wants to merge 1 commit into
Ataraxy-Labs:mainfrom
Iron-Ham:Iron-Ham/cache-bench-graph
Closed

Reuse entity graph during bench runs#21
Iron-Ham wants to merge 1 commit into
Ataraxy-Labs:mainfrom
Iron-Ham:Iron-Ham/cache-bench-graph

Conversation

@Iron-Ham

Copy link
Copy Markdown
Contributor

Summary

  • Build a reusable repository entity graph for stable checkouts.
  • Use it in inspect bench so each commit only recomputes diff and scoring work.
  • Include one-time graph timing, file count, and entity count in bench output.

Closes #17

Test plan

  • git diff --check
  • rustfmt --check crates/inspect-core/src/analyze.rs crates/inspect-cli/src/commands/bench.rs
  • cargo test -p inspect-core analyze_with_reusable_graph_skips_per_commit_graph_build
  • cargo test --workspace
  • cargo run -p inspect-cli -- bench --repo . --limit 3

@vercel

vercel Bot commented May 20, 2026

Copy link
Copy Markdown

@Iron-Ham is attempting to deploy a commit to the rs545837's projects Team on Vercel.

A member of the Team first needs to authorize it.

@inspect-review inspect-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inspect review

Triage: 22 entities analyzed | 0 critical, 0 high, 16 medium, 6 low
Verdict: standard_review

Findings (3)

  1. [low] In analyze_with_options_and_graph, the function uses graph.file_count for timing but this represents the file count from when the graph was built, not the current analysis. This creates inconsistent timing data where file_count doesn't match the actual files processed in the current diff analysis.
  2. [low] In analyze_added_function, the code attempts to read dep.start_line.saturating_sub(1) from file lines, but if start_line is 0, saturating_sub(1) returns 0, which would incorrectly access the first line instead of returning None for an invalid line number.
  3. [low] In analyze_added_function, the code attempts to get line content with lines.get(dep.start_line.saturating_sub(1)) but start_line is 1-indexed while array access is 0-indexed. If start_line is 0 (which shouldn't happen but isn't validated), saturating_sub(1) returns 0, accessing line 1 instead of erroring, causing off-by-one access.

Reviewed by inspect | Entity-level triage found 0 high-risk changes

@Iron-Ham Iron-Ham marked this pull request as ready for review May 20, 2026 23:32
@Iron-Ham Iron-Ham force-pushed the Iron-Ham/cache-bench-graph branch from 7a0edc1 to fd22ee7 Compare May 21, 2026 05:30

@inspect-review inspect-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inspect review

Triage: 22 entities analyzed | 0 critical, 0 high, 16 medium, 6 low
Verdict: standard_review

Findings (3)

  1. [low] In analyze_with_options_and_graph, the function uses graph.file_count for timing but this represents the file count from when the graph was built, not the current analysis. This creates inconsistent timing data where file_count doesn't match the actual files processed in the current diff analysis.
  2. [low] In analyze_added_function, the code attempts to read a line at index dep.start_line.saturating_sub(1) but this can still be out of bounds if start_line is 0 (after saturation) or if the file has fewer lines than start_line. The get() will return None but there's no validation that start_line is valid before attempting access.
  3. [low] In analyze_added_function, the code attempts to read dep.start_line.saturating_sub(1) from file lines, but if dep.start_line is 0, this will try to access line index -1 (wraps to usize::MAX with saturating_sub), causing an out-of-bounds access or returning None incorrectly.

Reviewed by inspect | Entity-level triage found 0 high-risk changes

@Iron-Ham Iron-Ham force-pushed the Iron-Ham/cache-bench-graph branch from fd22ee7 to 2c9841f Compare May 22, 2026 17:29

@inspect-review inspect-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inspect review

Triage: 23 entities analyzed | 0 critical, 0 high, 17 medium, 6 low
Verdict: standard_review

Findings (3)

  1. [low] In analyze_with_options_and_graph, the function passes repo_path as source_root to analyze_changes_with_graph, but repo_path is the repository root while the graph was built from a potentially different source tree (e.g., a specific commit's tree). This mismatch will cause collect_dependent_code to read files from the wrong location when include_dependent_code is enabled.
  2. [low] The build_analysis_graph function builds a graph from the current working tree of the repository, but when analyzing historical commits with analyze_with_graph, the graph entities may not match the entities at that commit. This will cause incorrect dependency analysis and risk scoring for historical commits.
  3. [low] In build_analysis_graph_from_files, the function signature takes source_root: &Path but the function returns AnalysisGraph which doesn't store the source_root. However, analyze_changes_with_graph needs the correct source_root to call collect_dependent_code. When using a prebuilt graph with analyze_with_graph, the wrong path (repo_path instead of the commit-specific checkout) will be used.

Reviewed by inspect | Entity-level triage found 0 high-risk changes

@Iron-Ham Iron-Ham closed this Jun 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid rebuilding the full entity graph for every bench commit

1 participant