Add CachePilot KVCache benchmark and evaluation suite

[imperson123]

This PR adds CachePilot, a non-invasive KVCache benchmark and evaluation suite for Mooncake.

CachePilot provides:

• a wrapper for Mooncake official store_kv_bench.py;
• real Mooncake Store verify_write and read_perf benchmark support;
• prefix reuse workload evaluation;
• retrieval scheduler strategy evaluation;
• CSV/log/figure generation and Streamlit dashboard.

The benchmark does not modify Mooncake core components and is placed under benchmarks/cachepilot.

Basic usage

cd benchmarks/cachepilot
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
bash scripts/run_all.sh

[gemini-code-assist]

Code Review

This pull request introduces CachePilot, a non-invasive evaluation suite for Mooncake's KVCache. It includes wrappers for running official store benchmarks, prefix cache reuse simulations, retrieval scheduler simulations, and a Streamlit dashboard for visualization. The feedback highlights a hardcoded Python path in the Windows batch script that hinders portability, as well as logic bugs and redundant code in the store runner script regarding the default object count handling for the write verification scenario.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The script contains a hardcoded absolute path to D:\\python.exe when creating the virtual environment. This is highly specific to a single local machine and will fail on almost all other Windows environments. It is recommended to use the standard python command instead.

  echo [.venv not found] Creating with python ...\n  python -m venv .venv

[gemini-code-assist]

There is a logic bug in how nr_objects is handled for the verify_write scenario. The code uses pass when args.nr_objects == 64, which does nothing and leaves nr_objects as 64. Combined with the unconditional overwrite on lines 387-390, the intended default of 16 is never applied when running the script directly without explicitly passing --nr-objects. Setting nr_objects = 16 here fixes this issue.

    nr_objects = args.nr_objects\n    if args.scenario == \"verify_write\" and args.nr_objects == 64:\n        # Default to 16 for verify_write if not explicitly overridden\n        nr_objects = 16

[gemini-code-assist]

With the fix applied to lines 361-364 to correctly set nr_objects = 16 as the default for verify_write, this block can be simplified to directly assign effective_nr = nr_objects, removing the redundant and buggy conditional overwrite.

Suggested change

	effective_nr = 16 if args.scenario == "verify_write" else nr_objects
	# Allow explicit --nr-objects to win when user set it in verify script
	if args.scenario == "verify_write":
	effective_nr = args.nr_objects
	effective_nr = nr_objects