Choose the committed-state publication protocol

[fproulx-boostsecurity]

Question

How should Pantry and GraphHub represent and publish one revisioned committed-state signal after successful Analysis ingestion so graph clients refresh from a snapshot and cannot observe candidate deltas, while ordinary Pantry mutations retain granular observer events?

Resolve revision semantics, observer interface shape, interaction with queued delta batches, client version handling, reconnect and stale-client behavior, and the absence of publication on serialization or persistence failure.

[fproulx-boostsecurity]

Resolution

Choose a durable, persistence-first Pantry publication protocol.

Committed state and revision

• A Pantry revision is a durable uint64 identifying one committed Pantry state. A new empty Pantry begins at revision zero.
• Revision is serialized in the same schema 3.0 Pantry snapshot as Assets and Relationships.
• One changed durable logical operation advances revision once. No-op and failed operations do not advance it.
• Every Kitchen Pantry write is persistence-first. CommittedState.Update publishes granular changes for ordinary operations; CommittedState.Replace publishes a bulk refresh marker for Analysis ingestion. Both use the transaction semantics selected in Prototype the Pantry committed-state transaction interface #102.

Observer seam

type Observer interface {
    OnPantryChange(ChangeSet)
}

type ChangeSet struct {
    Revision Revision
    Change   PublishedChange
}

PublishedChange is a typed variant:

• GranularChanges contains the final old-to-new Asset and Relationship changes for one ordinary durable operation.
• CommittedState contains no candidate deltas and requires snapshot refresh.

Candidate construction emits nothing. Publication is ordered after successful persistence and live replacement but before the shared writer gate opens.

GraphHub protocol

• GraphHub may batch contiguous granular change sets. A delta carries base_revision and resulting revision.
• A full-mode client applies a delta only when its current revision equals base_revision, ignores a stale duplicate whose resulting revision is not newer, and requests a snapshot for any other mismatch.
• A CommittedState marker fences the stream: GraphHub cancels and discards every pending earlier granular delta, then sends snapshot_required with the minimum revision.
• Filtered and automatically filtered clients request a snapshot for every newer change rather than applying deltas locally.
• Every graph projection is built from one captured Pantry snapshot containing one complete state and revision.

Client recovery

• A refreshing client retains its last complete graph, coalesces refresh requests, stops applying deltas, and accepts only a snapshot meeting its newest minimum revision.
• GraphHub never silently drops an ordered state message. A full outbound client queue disconnects that client.
• Reconnection always starts with a current mode-specific snapshot.
• GraphHub retains no delta replay history, revision acknowledgements, or per-client revision state.

Serialization, pre-store cancellation, or persistence failure emits no revision or publication. Cancellation after successful persistence cannot turn the committed operation into a reported failure.

The durability finding amends #102 as recorded at #102 (comment).

Decision source:

• Branch: https://github.com/boostsecurityio/smokedmeat/tree/plan/analysis-ingestion-implementation
• Commit: 4b91380

Maintainer confirmed shared understanding on 2026-07-30.