Release parse-stack-next v5.0.0

Major release preparing parse-stack-next v5.0.0 with API, tooling, and infra updates. Renames the gem to parse-stack-next and updates docs/README and changelog. Introduces an ergonomic Redis cache with a ConnectionPool-backed Pool wrapper and optional cache namespacing, plus ActiveSupport cache events and graceful pool-timeout handling. Adds vector/embeddings and vector-search primitives, a GraphQL type generator for Parse::Object models (opt-in), and many model/vector/graphQL helpers. Improves Ruby 3 compatibility by replacing class variables with class-instance state guarded by mutexes/monitors, refactors LiveQuery request id generation, and tightens various validations. MCP transport and agent code add health checks, protocol-version validation, and structured tool output. Also adds MongoDB instrumentation events and a slow-query subscriber hook. Includes CI docs workflow (YARD) and a large suite of new/updated tests to cover the new features and integrations.

Author: AdrianCurtin

.github/workflows/docs.yml

@@ -0,0 +1,39 @@
+name: Publish YARD Docs
+
+on:
+  push:
+    branches: [master]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: '3.4'
+          bundler-cache: true
+      - name: Build YARD docs
+        run: bundle exec rake yard
+      - uses: actions/upload-pages-artifact@v3
+        with:
+          path: doc/parse-stack
+  deploy:
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - id: deployment
+        uses: actions/deploy-pages@v4

CHANGELOG.md

@@ -1,7 +1,7 @@
 source "https://rubygems.org"
 
-# Specify your gem's dependencies in parse-stack.gemspec
-gemspec
+# Specify your gem's dependencies in parse-stack-next.gemspec
+gemspec name: "parse-stack-next"
 
 group :test, :development do
   gem "dotenv"

Gemfile

@@ -1,9 +1,10 @@
 PATH
   remote: .
   specs:
-    parse-stack (4.4.3)
+    parse-stack-next (5.0.0)
       activemodel (>= 6.1, < 9)
       activesupport (>= 6.1, < 9)
+      connection_pool (>= 2.2, < 4)
       csv (~> 3.3)
       faraday (~> 2.0)
       faraday-net_http_persistent (~> 2.0)
@@ -50,11 +51,16 @@ GEM
       faraday-net_http (>= 2.0, < 3.5)
       json
       logger
-    faraday-net_http (3.4.2)
+    faraday-net_http (3.4.3)
       net-http (~> 0.5)
     faraday-net_http_persistent (2.3.1)
       faraday (~> 2.5)
       net-http-persistent (>= 4.0.4, < 5)
+    fiber-storage (1.0.1)
+    graphql (2.6.2)
+      base64
+      fiber-storage
+      logger
     i18n (1.14.8)
       concurrent-ruby (~> 1.0)
     io-console (0.8.2)
@@ -155,11 +161,12 @@ PLATFORMS
 DEPENDENCIES
   debug (>= 1.0)
   dotenv
+  graphql (~> 2.0)
   minitest
   minitest-mock
   minitest-reporters
   mongo
-  parse-stack!
+  parse-stack-next!
   pry
   puma
   rack-test

Gemfile.lock

@@ -285,7 +285,7 @@ Cooperative cancellation lets clients abort an in-flight long-running tool call.
 
 2. **SSE client disconnect.** When the underlying TCP connection closes (browser tab closed, network drop), Rack calls `SSEBody#close`, which trips the same cancellation token.
 
-**Identity binding (required for `notifications/cancelled`).** The cancelling request **must** carry the same `X-MCP-Session-Id` header as the original request. The header is sanitized into `agent.correlation_id` and used as half of the registry key (the JSON-RPC `requestId` is the other half). Cancellation without a matching `X-MCP-Session-Id` is a silent no-op — this prevents an attacker who guesses sequential JSON-RPC ids from cancelling other clients' in-flight requests. Failures (no session id, no matching entry, mismatched session id) all return `202` so the response shape is not a probe oracle.
+**Identity binding (required for `notifications/cancelled`).** The cancelling request **must** carry the same `Mcp-Session-Id` header as the original request. The header is sanitized into `agent.correlation_id` and used as half of the registry key (the JSON-RPC `requestId` is the other half). Cancellation without a matching `Mcp-Session-Id` is a silent no-op — this prevents an attacker who guesses sequential JSON-RPC ids from cancelling other clients' in-flight requests. Failures (no session id, no matching entry, mismatched session id) all return `202` so the response shape is not a probe oracle.
 
 **Cooperative checkpoints.** Cancellation is observed at safe points inside tool execution, not by forcibly killing the dispatcher thread. The two checkpoints built into `Parse::Agent#execute` are:
 
@@ -319,7 +319,7 @@ The stream still emits the `response` SSE event before closing so clients do not
 
 **Scope and limitations.**
 - The cancellation registry is per `MCPRackApp` instance. Cancellation does not span multiple mount points within a process, nor multiple processes in a clustered deployment.
-- Clients that do not set `X-MCP-Session-Id` lose cancellation but keep every other MCP feature.
+- Clients that do not set `Mcp-Session-Id` lose cancellation but keep every other MCP feature.
 - The standalone WEBrick-backed `MCPServer` does not support streaming and therefore does not support cancellation; calls return a single buffered response with no opportunity to interrupt.
 
 ---
@@ -1834,7 +1834,7 @@ Every tool call dispatched through `Agent#execute` fires the `"parse.agent.tool_
 
 **Conversation correlation across multi-tool sessions.** Without correlation, individual tool-call events have no link between them — a Datadog dashboard sees "user X did query_class" and "user X did get_object" as independent points, with no way to know they belong to the same LLM turn. The dispatcher threads an optional correlation id through to every notification:
 
-- **Header path (recommended for hosted MCP):** the client sends `X-MCP-Session-Id: <opaque-id>` on every request in the conversation. `MCPRackApp` reads the header, sanitizes the value (charset `[A-Za-z0-9._-]`, max 128 chars — anything else is silently dropped to prevent log injection), and sets `agent.correlation_id` unless the factory has already supplied one. Notifications fired during that request carry the value as `payload[:correlation_id]`.
+- **Header path (recommended for hosted MCP):** the client sends `Mcp-Session-Id: <opaque-id>` on every request in the conversation (the MCP 2025-06-18 Streamable HTTP spec-canonical name). `MCPRackApp` reads the header, sanitizes the value (charset `[A-Za-z0-9._-]`, max 128 chars — anything else is silently dropped to prevent log injection), and sets `agent.correlation_id` unless the factory has already supplied one. Notifications fired during that request carry the value as `payload[:correlation_id]`.
 
 - **Factory path (for application-bound sessions):** application code that already has an internal session identifier can override the client-supplied header by setting it inside the agent factory:
 
@@ -1853,7 +1853,7 @@ Every tool call dispatched through `Agent#execute` fires the `"parse.agent.tool_
 
 When unset (no header, no factory assignment), `payload[:correlation_id]` is omitted entirely — the key does not appear in the payload hash.
 
-The same `X-MCP-Session-Id` header is **required** for cooperative cancellation via `notifications/cancelled` — see the Cancellation section. Clients that thread the header through every request in a conversation get both correlated audit logs and cancellation; clients that don't lose both but keep every other MCP feature.
+The same `Mcp-Session-Id` header is **required** for cooperative cancellation via `notifications/cancelled` — see the Cancellation section. Clients that thread the header through every request in a conversation get both correlated audit logs and cancellation; clients that don't lose both but keep every other MCP feature.
 
 **Cancellation notification asymmetry.** A tool cancelled BEFORE it runs (via `agent.cancelled?` at the dispatcher's first checkpoint) does not fire `parse.agent.tool_call` — the tool never executed, so there is nothing to instrument. This matches how rate-limit and permission refusals are surfaced. A tool cancelled AFTER it returns (second checkpoint, "client cancelled while the tool's I/O was running") DOES fire the notification with `success: false, error_code: :cancelled`. Subscribers that count cancellations should expect the second shape; pre-run cancellations are visible to operators only via the wire response.
 

README.md

@@ -0,0 +1,5 @@
+# encoding: UTF-8
+# frozen_string_literal: true
+
+# Auto-required entry point for the `parse-stack-next` gem.
+require_relative "./parse/stack.rb"

docs/mcp_guide.md

@@ -604,7 +604,7 @@ def assert_llm_endpoint_allowed!(endpoint)
 
     # @return [String, nil] caller-supplied identifier that ties multiple
     #   tool calls into a single logical conversation. Set by the transport
-    #   layer (MCPRackApp reads X-MCP-Session-Id) or directly by an
+    #   layer (MCPRackApp reads Mcp-Session-Id) or directly by an
     #   embedder. Included in every `parse.agent.tool_call` notification
     #   payload as `:correlation_id` when present. Sanitized to a max of
     #   128 characters from the set `[A-Za-z0-9._-]` to prevent log

lib/parse-stack-next.rb

@@ -188,14 +188,30 @@ def self.strip_underscore_smuggled_headers!(env)
       #   CSRF and force a CORS preflight on browser `fetch()`, so this
       #   gate closes the browser-driven attack surface entirely. Pair
       #   with `allowed_origins` for defense in depth.
+      # @param health_path [String, nil] when set (e.g. `"/health"`),
+      #   `GET` requests to that exact path return `200 {"status":"ok"}`
+      #   without invoking the agent_factory, without authentication,
+      #   without rate-limiting, and without applying the
+      #   `allowed_origins` / `require_custom_header` CSRF gates.
+      #   Intended as a liveness probe for load balancers and
+      #   orchestrators (Kubernetes, ECS, Consul) that cannot present a
+      #   matching `Origin` or custom header. Because the probe sits
+      #   ahead of the pre-auth rate limiter, operators should
+      #   front-edge rate-limit the path at the LB/Nginx layer if
+      #   public-facing. The response intentionally contains no
+      #   version, build, or counter information — fingerprint-minimal
+      #   by design. `nil` (default) disables the endpoint entirely;
+      #   empty-string values are coerced to `nil`. Any non-GET method
+      #   on the path falls through to the standard 405 handler.
       # @raise [ArgumentError] if both or neither of agent_factory/block are given.
       def initialize(agent_factory: nil, max_body_size: DEFAULT_MAX_BODY_SIZE,
                      logger: nil, streaming: false,
                      heartbeat_interval: DEFAULT_HEARTBEAT_INTERVAL,
                      max_concurrent_dispatchers: nil,
                      pre_auth_rate_limiter: nil,
                      allowed_origins: nil,
-                     require_custom_header: nil, &block)
+                     require_custom_header: nil,
+                     health_path: nil, &block)
         if agent_factory && block
           raise ArgumentError, "Provide agent_factory: OR a block, not both"
         end
@@ -215,6 +231,7 @@ def initialize(agent_factory: nil, max_body_size: DEFAULT_MAX_BODY_SIZE,
         @pre_auth_rate_limiter      = pre_auth_rate_limiter
         @allowed_origins            = normalize_allowed_origins(allowed_origins)
         @required_custom_header     = normalize_required_custom_header(require_custom_header)
+        @health_path                = health_path.is_a?(String) && !health_path.empty? ? health_path : nil
         # Per-app registry of in-flight cancellable requests. Keyed by
         # [correlation_id, request_id]. A `notifications/cancelled` POST
         # whose `params.requestId` matches an entry trips the registered
@@ -267,6 +284,15 @@ def call(env)
         #    underscored form collapses-and-overwrites the trusted slot.
         self.class.strip_underscore_smuggled_headers!(env)
 
+        # 0a. Liveness probe. When `health_path:` is configured, a GET to
+        #     that exact path returns `{"status":"ok"}` without auth,
+        #     rate-limiting, or factory invocation. Intentionally
+        #     fingerprint-minimal: no version, no build, no counter —
+        #     a load balancer needs "is it up?", not "what is it?".
+        if @health_path && env["PATH_INFO"] == @health_path && env["REQUEST_METHOD"] == "GET"
+          return [200, json_headers, ['{"status":"ok"}']]
+        end
+
         # 0b. NEW-MCP-6: pre-auth rate limit. Runs BEFORE the agent_factory
         #     so a malformed body / missing key / empty `{}` cannot force
         #     the operator-supplied factory to round-trip to Parse Server
@@ -348,6 +374,32 @@ def call(env)
           return [400, json_headers, [json_rpc_error(-32_600, "Invalid Request")]]
         end
 
+        # 4c. MCP-Protocol-Version header validation (MCP 2025-06-18
+        #     Streamable HTTP). The spec says:
+        #       - The client MUST send `MCP-Protocol-Version: <ver>`
+        #         on every request AFTER initialize.
+        #       - If absent on a non-initialize request, the server
+        #         SHOULD assume `2025-03-26` for backwards compatibility.
+        #       - If present but not a version the server supports,
+        #         the server MUST respond `400 Bad Request`.
+        #     Initialize requests are exempt — initialize IS the
+        #     negotiation, so the header is meaningless there.
+        #     Cancellation notifications are also exempt because they
+        #     may be sent by a client that has not (yet) completed
+        #     initialize against this transport instance (e.g. a
+        #     reconnecting client cancelling a pre-disconnect request).
+        unless body["method"] == "initialize" ||
+               body["method"] == "notifications/cancelled"
+          requested = env["HTTP_MCP_PROTOCOL_VERSION"]
+          if requested.is_a?(String) && !requested.empty? &&
+             !Parse::Agent::MCPDispatcher::SUPPORTED_PROTOCOL_VERSIONS.include?(requested)
+            return [400, json_headers,
+                    [json_rpc_error(-32_600,
+                                    "Unsupported MCP-Protocol-Version: #{requested}",
+                                    id: body["id"])]]
+          end
+        end
+
         # 5. Agent factory — auth gate. Rescue Unauthorized first, then catch-all
         #    for unexpected factory errors.
         begin
@@ -363,24 +415,27 @@ def call(env)
           return [500, json_headers, [json_rpc_error(-32_603, "Internal error")]]
         end
 
-        # 5b. Thread the conversation correlation id through. Source:
-        #     X-MCP-Session-Id header. Only fills it when the factory
-        #     hasn't already assigned one — application code that needs to
-        #     override the client-supplied id (e.g., bind to an internal
-        #     session record) can do so in the factory and we don't
-        #     stomp on it. The Parse::Agent#correlation_id= setter
-        #     sanitizes the value; an invalid header is silently dropped.
+        # 5b. Thread the conversation correlation id through. Source
+        #     header: the MCP 2025-06-18 Streamable HTTP spec-canonical
+        #     `Mcp-Session-Id` (Rack env key `HTTP_MCP_SESSION_ID`).
+        #
+        #     Only fills it when the factory hasn't already assigned one
+        #     — application code that needs to override the
+        #     client-supplied id (e.g., bind to an internal session
+        #     record) can do so in the factory and we don't stomp on it.
+        #     The Parse::Agent#correlation_id= setter sanitizes the
+        #     value; an invalid header is silently dropped.
         if agent && agent.respond_to?(:correlation_id=) &&
            agent.correlation_id.nil? &&
-           (sid = env["HTTP_X_MCP_SESSION_ID"])
+           (sid = env["HTTP_MCP_SESSION_ID"])
           agent.correlation_id = sid
         end
 
         # 5c. notifications/cancelled — special-cased BEFORE the dispatcher.
         #     A JSON-RPC notification has no `id`, expects no response
         #     body, and must trip the in-flight request whose
         #     `(correlation_id, request_id)` matches. We require the
-        #     cancelling request to carry the same X-MCP-Session-Id
+        #     cancelling request to carry the same Mcp-Session-Id
         #     (sanitized into agent.correlation_id above) as the original
         #     request — otherwise an attacker who guesses sequential
         #     JSON-RPC ids could cancel arbitrary in-flight requests.
@@ -945,7 +1000,7 @@ def build_error_envelope(error)
       # matching CancellationToken.
       #
       # Identity binding: cancellation requires the cancelling request's
-      # `X-MCP-Session-Id` (sanitized into `agent.correlation_id`) to
+      # `Mcp-Session-Id` (sanitized into `agent.correlation_id`) to
       # match the original request's. This prevents an attacker who
       # guesses sequential JSON-RPC request ids from cancelling other
       # clients' in-flight requests. A registration with a nil

lib/parse/agent.rb

@@ -51,13 +51,22 @@ def initialize(message, stage: nil, reason: nil, operator: nil)
       # Validate an aggregation pipeline for security issues.
       # Delegates to {Parse::PipelineSecurity.validate_pipeline!} and
       # translates its error into {PipelineSecurityError} for backwards
-      # compatibility.
+      # compatibility. Additionally refuses Atlas-stage-0-only operators
+      # (`$search`, `$searchMeta`, `$vectorSearch`, `$listSearchIndexes`)
+      # which are legal SDK-emitted stages but must NOT appear in a
+      # caller-supplied agent pipeline — the agent surface for those is
+      # the dedicated `atlas_search` / `semantic_search` tools, and the
+      # Agent's tenant-scope `$match` prepend would push them off
+      # stage 0 anyway. See
+      # {Parse::PipelineSecurity::STAGE0_ONLY_ATLAS_STAGES}.
       #
       # @param pipeline [Array<Hash>] the aggregation pipeline stages
       # @raise [PipelineSecurityError] if pipeline contains blocked or unknown stages
       # @return [true] if pipeline is valid
       def validate!(pipeline)
         Parse::PipelineSecurity.validate_pipeline!(pipeline)
+        refuse_stage0_only_atlas_stages!(pipeline)
+        true
       rescue Parse::PipelineSecurity::Error => e
         raise PipelineSecurityError.new(
           e.message,
@@ -67,6 +76,24 @@ def validate!(pipeline)
         )
       end
 
+      # @api private
+      def refuse_stage0_only_atlas_stages!(pipeline)
+        return unless pipeline.is_a?(Array)
+        pipeline.each do |stage|
+          next unless stage.is_a?(Hash)
+          stage.each_key do |k|
+            key = k.to_s
+            next unless Parse::PipelineSecurity::STAGE0_ONLY_ATLAS_STAGES.include?(key)
+            raise PipelineSecurityError.new(
+              "Stage #{key} is not allowed in caller-supplied agent pipelines. " \
+              "Use the dedicated atlas_search / semantic_search agent tool instead.",
+              stage: key,
+              reason: :stage0_only_atlas_stage,
+            )
+          end
+        end
+      end
+
       # Check if a pipeline is valid without raising.
       #
       # @param pipeline [Array<Hash>] the aggregation pipeline