[Benchmarks] AgentCube SnapStart Validation for Agentic RL Rollouts

[Abhinav-kodes]

(Updated) - 30/05/2026

This issue tracks benchmark scenarios for validating the SnapStart direction in #366 under agentic RL rollout workloads.

The goal is to measure whether the Kuasar-based SnapStart path can meet AgentCube’s latency, concurrency, and fallback requirements for rollout-heavy use cases.

Benchmark scenarios

• N-way concurrent session creation.
• Cold start vs snapshot restore latency.
• Restore failure and cold-start fallback behavior.
• Memory footprint and throughput under burst load.

Why this matters

Agentic RL rollouts can require many fast environment launches, resets, and evaluations. This issue defines the validation work needed to check whether SnapStart is a good fit for that workload.

Relationship to #366

This issue is now a supporting benchmark and validation tracker for #366, not a separate backend proposal.

[Abhinav-kodes]

cc @hzxuzhonghu

[Abhinav-kodes]

@acsoto your thoughts please

[acsoto]

Thanks for writing this up. I think this proposal is aligned with the broader direction in #267

That said, after looking at #366, I think the immediate implementation direction should probably align with the SnapStart design rather than introducing a separate direct-to-Firecracker backend first.

The main reason is architectural fit. AgentCube’s current control plane is Kubernetes-native: runtimes are represented as CodeInterpreter / AgentRuntime, sessions are created through Sandbox / SandboxClaim, and the router depends on Workload Manager session metadata. #366 keeps that model intact by layering an independent SnapStart CRD on top of existing runtimes, using Kuasar WarmForkSnapshot where available, and falling back to cold start when a snapshot is unavailable.

The Firecracker proposal is still useful as a motivation and as a future comparison point, especially for RL fan-out latency targets. But I would avoid splitting the implementation into a separate direct runtime backend before the SnapStart path is validated. A better near-term path may be:

1. Review and refine the SnapStart design in docs: add agentcube snapstart proposal #366.
2. Define benchmark scenarios that come from [Proposal] Position AgentCube as a Stateful, Isolated, Concurrent Rollout Execution Layer for Agentic RL and Verifiable Agentic Tasks #267 / [Benchmarks] AgentCube SnapStart Validation for Agentic RL Rollouts #365, such as N-way rollout fan-out, cold start vs snapshot restore, and restore failure fallback.
3. Implement the first SnapStart PoC for CodeInterpreter on Kuasar.
4. Revisit Firecracker/direct-runtime support later if benchmarks show Kuasar SnapStart or the Kubernetes-native path cannot meet the rollout latency goals.

So I’m +1 on the problem statement and RL rollout motivation, but I’d suggest treating this issue as aligned with #366 rather than starting a competing backend design.