Prototype the Pantry committed-state transaction interface

[fproulx-boostsecurity]

Question

What Pantry-level interface should provide candidate snapshot construction, a writer gate covering every mutator, serialization and validation, persistence through an adapter, and non-failing in-place replacement of live internals while keeping the Pantry pointer stable?

Compare internal seam options with a cheap prototype. Resolve old-or-new reader visibility, writer participation, failure atomicity, version behavior, adapter responsibilities, and how asynchronous metadata-sync and other ordinary writers compose with reconciliation. Do not implement production behavior.

[fproulx-boostsecurity]

Prototype ready for the Pantry transaction interface decision.

Asset:

• Branch: https://github.com/boostsecurityio/smokedmeat/tree/prototype/pantry-committed-state-transaction
• Commit: 7ed0e5d
• Run locally: make prototype-pantry-transaction

The prototype drives reader sampling, a waiting metadata-sync writer, semantic no-op, successful persistence, persistence rollback, cancellation before persistence, cancellation after persistence, and capability leakage through three seams:

1. Raw callback: Pantry.UpdateCommitted(ctx, store, func(*Pantry) error). Smallest interface, but persistence is supplied per call and the raw candidate lifetime is enforced only by convention.
2. Explicit transaction: BeginWrite, Candidate, Commit, Abort. Flexible enough for other multi-Asset writers, but callers choose publication mode, manage gate lifetime, and can block every writer by leaking a capability.
3. Bound module with Draft: CommittedState.Update(ctx, func(*Draft) error). Safest common call, but duplicates Pantry reading and editing methods on a second Draft interface.

Recommendation: combine 1 and 3. Bind the stable live Pantry and SnapshotStore once, retain a single callback operation, and pass an isolated observer-free *Pantry candidate instead of adding a second graph-editing interface:

type SnapshotStore interface {
    Replace(serialized []byte) error
}

type CommittedState struct {
    // stable live Pantry, store, and private collaborators
}

func NewCommittedState(live *Pantry, store SnapshotStore) *CommittedState

func (s *CommittedState) Update(
    ctx context.Context,
    update func(candidate *Pantry) error,
) error

Required semantics:

• Update acquires one context-aware writer gate shared by every Pantry mutator, then clones the latest committed state.
• Readers continue on the old state. Reads needing several graph queries from one revision use one captured read snapshot.
• Candidate mutations are isolated, defensively copied, and observer-free. The trusted callback must not retain the candidate or perform external side effects.
• An unchanged candidate is a successful no-op with no persistence, version change, or publication.
• Changed candidates are validated and serialized exactly once, followed by a final cancellation check.
• SnapshotStore.Replace is deliberately non-cancellable. Error proves prior durable bytes remain authoritative; nil means the supplied bytes are durable.
• After persistence succeeds, live internals are replaced through the stable Pantry pointer with no fallible work, the live version advances exactly once, and one committed-state signal is ordered before the writer gate opens.
• Cancellation after persistence cannot turn a commit into a reported failure.
• A queued ordinary writer then runs against the new state and retains granular publication. On rollback or pre-store cancellation, it runs against the old state.
• BBolt owns only atomic byte replacement. Candidate construction, validation, serialization, replacement, revisioning, and publication stay inside Pantry.
• The committed revision wire and restart semantics remain for Choose the committed-state publication protocol #100; Prototype the Pantry committed-state transaction interface #102 establishes only one live version transition per changed commit.

This ticket remains open pending maintainer reaction to the prototype.

[fproulx-boostsecurity]

Resolution

Choose a concrete Pantry committed-state module with one callback operation:

type SnapshotStore interface {
    Replace(serialized []byte) error
}

type CommittedState struct {
    // stable live Pantry, store, and private collaborators
}

func NewCommittedState(live *Pantry, store SnapshotStore) *CommittedState

func (s *CommittedState) Update(
    ctx context.Context,
    update func(candidate *Pantry) error,
) error

This combines the smallest callback lifecycle with persistence bound once at construction. It deliberately rejects both an explicit caller-managed transaction capability and a duplicate Draft graph-editing interface.

The prototype and maintainer reaction establish these constraints:

• Update acquires one context-aware writer gate shared by every Pantry mutator, then clones the latest committed state. Metadata sync, purge, runtime evidence, and other ordinary writers wait at the same gate.
• Readers do not acquire the writer gate. They continue to observe the old committed state during candidate construction and persistence, then the new state after replacement. Multi-query consumers that require one revision use a captured read snapshot.
• The callback receives an isolated, observer-free *Pantry. It is a trusted internal callback: it must not retain the candidate or perform externally visible side effects. Pantry inputs and read results require defensive copying where maps or slices could bypass the gate.
• An unchanged candidate is a successful no-op with no persistence, live-version change, or publication.
• A changed candidate is validated and serialized exactly once. Context cancellation is checked once more before persistence begins.
• SnapshotStore.Replace is intentionally non-cancellable. An error guarantees prior durable bytes remain authoritative; nil means the supplied complete Pantry snapshot is durable. The BBolt adapter performs one atomic byte replacement and owns no graph logic.
• Persistence failure or pre-store cancellation discards the candidate, emits nothing, and leaves the previous live and durable Pantry authoritative.
• After persistence succeeds, context cancellation is ignored for that transaction. The prepared candidate internals replace the live internals through the stable *Pantry pointer with no fallible work between persistence and replacement.
• One changed commit advances the live Pantry version exactly once, regardless of candidate mutation count. Candidate-local mutations do not affect the live version. The committed revision wire and restart semantics remain for Choose the committed-state publication protocol #100.
• One committed-state signal is ordered after live replacement and before the writer gate opens. Its exact observer and GraphHub protocol remains for Choose the committed-state publication protocol #100. Ordinary writers retain granular events and cannot overtake the committed signal.
• Candidate construction is O(Assets + Relationships). Peak memory includes old live state, candidate state, and one serialized snapshot. Readers remain available; writers wait through candidate construction, validation, serialization, persistence, replacement, and signal ordering.

The AnalysisIngestor selected in #101 owns Analysis authority, ownership, identity, and preservation semantics. CommittedState owns only the generic Pantry transition from one committed state to the next.

Prototype asset: https://github.com/boostsecurityio/smokedmeat/tree/prototype/pantry-committed-state-transaction

Maintainer approved the recommended hybrid on 2026-07-29.

[fproulx-boostsecurity]

Durability amendment from #100

The publication-protocol review exposed a contradiction in the original single-operation resolution: ordinary Kitchen Pantry writers currently publish granular observer events before persistence, so a crash could make an observed revision disappear after restart.

Maintainer-approved correction:

func (s *CommittedState) Update(
    ctx context.Context,
    change func(candidate *Pantry) error,
) error

func (s *CommittedState) Replace(
    ctx context.Context,
    change func(candidate *Pantry) error,
) error

• Update is persistence-first and publishes one final granular change set for an ordinary durable logical operation.
• Replace is persistence-first and publishes one committed-state snapshot-refresh marker. Analysis ingestion uses it.
• Both operations share the same private writer gate, candidate, validation, single serialization, atomic persistence, rollback, stable-pointer replacement, revision, and publication machinery.
• One changed durable logical operation advances Pantry revision once, regardless of the number of candidate edits or granular changes.
• Persistence failure publishes nothing and leaves prior durable and live Pantry authoritative.

This comment supersedes only #102 wording that selected one public operation or allowed ordinary live publication before persistence. The candidate callback and transaction mechanics remain unchanged.

Protocol source: 4b91380