vMCP Code Mode

[jerm-dro]

Overview

Add opt-in Starlark script execution ("code mode") to vMCP, allowing agents to batch multi-tool workflows into a single server-side call — reducing round-trips, context bloat, and inference cycles. Based on the prototype in PR #4714.

Background

Agents today make sequential tool calls with model inference between each one. For multi-service workflows (e.g., incident triage across PagerDuty, Datadog, Slack, Jira), this means 10+ round-trips and significant token spend. Scripting is possible today only through CLI/shell access, which is less safe and unavailable for knowledge worker use cases where agents lack a shell environment. The prototype demonstrates ~15s improvement by executing a Starlark script server-side that calls tools directly, with parallel() fan-out — without requiring shell access.

This epic production-hardens the prototype and ships it as an opt-in feature. It does NOT implement the full RFC THV-0060 session initialization model.

User Story Breakdown

Story ID	Title	Description	Persona	Goal	Acceptance Criteria	GitHub Issue #
STORY-001	Ship opt-in code mode for vMCP	As a platform engineer, I want my agents to be able to execute scripts on tools without shell access so that they can safely reduce context bloat and inference cycles	Platform engineer	Enable server-side script execution for agents	Config toggle, proper dispatch, configurable limits, timeouts, optimizer compat
STORY-002	Add observability for script execution	As a cluster operator, I want logging and metrics for script execution so that I can monitor and diagnose issues	Cluster operator	Operational visibility	Structured logs, execution metrics, telemetry integration
STORY-003	Track code mode adoption and usage	As a ToolHive developer, I want to know how many people are opting in to code mode and how many tool calls come through it so that I can gauge adoption and prioritize investment	ToolHive developer	Understand adoption	Opt-in counts, call volume comparison, telemetry pipeline

Acceptance Criteria

• Code mode can be enabled per-VirtualMCPServer via config flag
• Agents can execute Starlark scripts with tool calls, loops, conditionals, and parallel()
• Script execution is bounded (step limits, concurrency caps, timeouts)
• Works correctly with optimizer enabled
• Observability: logs and metrics for script execution
• Adoption tracking: code mode opt-in and call volume metrics
• All linked user stories completed

References

• Prototype PR: Prototype Starlark script middleware for vMCP #4714
• RFC THV-0060: RFC-0060: Starlark Session Initialization for vMCP toolhive-rfcs#60
• Slack thread: https://stacklok.slack.com/archives/C0AMVK58NUD/p1775761162900059

[jerm-dro]

Intake Notes

Intake: vMCP Code Mode (Starlark Script Middleware)

Epic Summary

Production-harden the prototype execute_tool_script Starlark middleware (PR #4714) and ship it as an opt-in feature in vMCP. The prototype validates the execution model; this epic addresses known limitations to make it shippable.

Background & Motivation

Agents today make sequential tool calls with model inference between each one. For multi-service workflows (e.g., incident triage across PagerDuty, Datadog, Slack, Jira, GitHub, Confluence), this means 10+ round-trips and significant token spend. The Starlark middleware lets agents submit a single script that calls multiple tools server-side, with loops, conditionals, and parallel() fan-out — returning an aggregated result in one tool call. Demo shows ~15s improvement over sequential approach.

Scope

In Scope

• Address the 7 known limitations from the prototype (PR Prototype Starlark script middleware for vMCP #4714):

  1. Config toggle: Add opt-in flag to enable/disable the script middleware (boolean in VirtualMCPServer spec or vMCP server config)
  2. Inner tool call mechanism: Replace httptest.NewRecorder with a proper internal dispatch mechanism
  3. Step limit configuration: Make per-script step limit configurable (currently hardcoded 100K)
  4. Concurrency cap for parallel(): Add a configurable limit on goroutines spawned by parallel()
  5. Tool call timeouts: Add per-tool-call timeout within script execution
  6. Result unwrapping: Make {"result": value} structured content unwrapping robust (currently specific to mcp-go SDK behavior)
  7. Test coverage: Ensure adequate unit and integration test coverage for hardened code

• Production-quality code: error handling, logging, metrics

• Documentation: user-facing docs for the feature, config reference

Out of Scope

• Full RFC THV-0060 session initialization model (backends(), publish(), presets)
• SSE transport support (deferred to follow-up)
• Starlark script security sandbox beyond step limits (future work)
• Demo environment / dummy MCP servers (prototype-only artifacts)
• .claude/skills/ for incident triage (demo-only)

Technical Context

Architecture

• Middleware sits above authz in the vMCP middleware chain
• Injects execute_tool_script virtual tool into tools/list responses
• Intercepts execute_tool_script tool/call and executes Starlark script
• Tool calls within scripts become synthetic tool/call passed down the middleware stack
• parallel() builtin enables concurrent fan-out

Key Files (Prototype)

• pkg/script/engine.go — Starlark execution engine
• pkg/script/bridge.go — Tool bridge (MCP tools → Starlark callables, parallel())
• pkg/script/middleware.go — HTTP middleware (intercept, inject, dispatch)
• pkg/vmcp/server/server.go — ScriptMiddleware config field
• cmd/vmcp/app/commands.go — Wiring

Dependencies

• go.starlark.net — Starlark interpreter for Go
• Existing vMCP middleware chain, authz middleware
• MCP tool registry for dynamic tool discovery

Decisions

Decision	Choice	Rationale
Scope	Prototype hardening only	Ship quickly, validate with users, iterate
Opt-in mechanism	Config flag (boolean)	Minimal surface area, no breaking changes
SSE transport	Deferred	JSON-only covers primary use cases for now
Issue tracking	stacklok/toolhive with dedicated label	Easy filtering of related stories

Personas

• Platform engineer: Configures vMCP instances, needs clear opt-in toggle and sensible defaults
• Agent developer: Writes Starlark scripts via agents, needs reliable execution with clear error messages and timeouts
• Cluster operator: Monitors vMCP in production, needs observability into script execution (logs, metrics)

References

• Draft PR: Prototype Starlark script middleware for vMCP #4714
• RFC THV-0060: RFC-0060: Starlark Session Initialization for vMCP toolhive-rfcs#60
• Slack thread: https://stacklok.slack.com/archives/C0AMVK58NUD/p1775761162900059