Releases: NoKV-Lab/holt
Releases · NoKV-Lab/holt
Release list
v0.9.1
Fixed
- Journal stream initialization now fences ordinary writes before either
persistent anchor slot can become durable. An interrupted initialization
keeps the fence active and accepts only an exact retry with the same genesis
anchor. - Open, replay, and recovery scans now reject an ordinary logical WAL record
after an attached-stream checkpoint anchor. This detects a surviving Holt
0.9.0 interrupted-initialization gap instead of accepting it as part of the
recovery stream.
Changed
- Ordinary writes now use an atomic stream fence instead of locking the
journal tail while no attached stream is active. - Attached submissions now use encoder-sealed record metadata. They no longer
decode and copy a record before queueing it. Replay and recovery scans still
decode records and validate their CRCs.
Upgrade notes
- Holt 0.9.1 uses the same format-4 WAL as 0.9.0. Existing stores do not need a
format migration. - If Holt 0.9.0 returned an error from
initialize_journal_streamand the
process then accepted ordinary writes or ran a checkpoint, rebuild from
authoritative application state before using the attached stream. Holt
0.9.1 rejects mixed records that remain in the WAL, but it cannot reconstruct
or detect a recovery gap already folded into a checkpoint.
v0.9.0
Added
- Added
DB::initialize_journal_stream,
DB::atomic_with_journal_envelope,DB::journal_state, and
DB::journal_envelopes_after. One opaque application recovery envelope and
its guarded multi-tree mutation now share one WAL record and one CRC. - Added
JournalAnchor,JournalEnvelope, bounded paged scans, typed stale
cursor and anchor mismatch errors, and a 16 MiB encoded-record limit. - Added an explicit process-local stream for memory databases. It follows the
same ordering, fencing, paging, and checkpoint-floor rules but provides no
reopen or crash durability.
Changed
- Advanced the WAL to format 4. Its 4096-byte header contains two
independently checksummed checkpoint-anchor slots. Checkpoint writes and
syncs both slots before it truncates the retained record suffix. - After callers initialize an attached recovery stream, Holt rejects ordinary
logical writes before mutation. Callers must route changes through
DB::atomic_with_journal_envelope; failed guards and stale anchors append no
envelope. - WAL scans stabilize and flush the committed async-ring prefix before reading
the retained suffix. Checkpoint and attached append use one lock order so
neither can cross the other's anchor boundary.
Fixed
- Compaction now budgets the eight-byte routing node used by every reachable
EmptyRoot. Delete-heavy trees can retain empty child blobs, so treating the
sentinel as root-only undercounted the routing arena and could stop
compaction when the encoded layout crossed a page boundary.
Upgrade notes
- Writable open upgrades a header-only format-3 WAL to format 4. Holt replays
a nonempty format-3 WAL only in read-only mode. It rejects writable open
before changing files or cached state. - Before upgrading a nonempty format-3 store, open it with Holt 0.8.5 and call
DB::checkpoint(). You can also export it through a read-only 0.9.0 open
and install the image into a fresh 0.9.0 store. - To downgrade from format 4, call
DB::export_checkpoint(). Install the
image into a fresh 0.8.x store withDB::install_checkpoint().
v0.8.5
Added
DB::atomicnow wraps failures inError::Atomicwith an
AtomicErrorKindofDefinitelyNotAppliedbefore its first batch mutation
orOutcomeUnknownafter apply begins. The method keeps its existing
signature.- Exposed put and delete batches through
holt_tree_atomicin the C ABI.
The wrapper validates every operation before it callsTree::atomic. - Added existing-only read-only opens through
AccessMode,
TreeConfig::read_only,TreeBuilder::read_only, and
holt_tree_open_read_only. Read-only handles replay the WAL in memory and
reject mutations and maintenance. - Added shared reader locks for file-backed stores. Writers keep an exclusive
lock, so multiple readers can share one store without overlapping a writer.
v0.8.4
Added
AtomicBatchandDBAtomicBatchnow provide a read-only
assert_absentguard. The guard publishes no marker, appends no WAL
operation, and consumes no record version. A failed guard aborts the
complete batch without publishing its mutations.
Fixed
Tree::atomicandDB::atomicnow reject oversized WAL records before
they reserve record versions or apply walker mutations. A capacity error
leaves the WAL, visible keys, and dirty state unchanged.
v0.8.3
Fixed
- File-store blob rewrites are now crash-safe. 0.8.2 rewrote a
same-GUID blob in place over the slot the durable manifest still
referenced. A 512 KiB framepwriteis not power-loss atomic, so a
crash mid-rewrite could tear the only complete copy of a checkpoint
base frame — the image the sync WAL needs for redo — losing or
corrupting acknowledged writes. Every rewrite now shadow-writes to a
fresh slot and publishes the remap first; the superseded slot
becomes reusable only after the manifest delta is durable and
in-flight readers have drained. Sizing note: a store whose every
live blob is rewritten between two flushes now peaks at roughly
twice its steady-state slot count; explicitvacuumreclaims the
tail. - Failed writes roll back only their own reserved slots. Slots
superseded by earlier writes in the same round stay pending until
the durability fence passes, instead of becoming reusable early on
the rollback path.
Added
- napi-rs Node.js bindings under
crates/holt-node, including the
multi-tree database API, with storage operations kept off the
event loop.
v0.8.2
Added
- Added explicit
Tree::vacuumandDB::vacuummaintenance APIs.gc
still performs logical reachability reclamation;vacuumfollows it with a
checkpoint and physically trims trailing reusable slots fromblobs.dat,
read.idx, andvalue.seg. - Added store-space observability for physical allocated bytes, tail
reclaimable slots/bytes, reusable middle slots, and vacuum relocation
counters. - Added GC observability for current orphan backlog, cumulative physical
reclamation, and the deferred count from the most recent full reachability
sweep. The latter is intentionally not reset by exact FIFO reclaim.
Changed
- Breaking stats API hardening.
TreeStats,DBStats, andOpenStatsare
now#[non_exhaustive];TreeStats/DBStatsadd GC lifecycle fields and
OpenStatsadds DB epoch-recovery duration. External code that constructed
these returned telemetry structs or destructured them without..must use
field access/non-exhaustive matching instead. This one-time break makes
future telemetry additions semver-compatible without compatibility shims. - GC Prometheus names follow their actual contracts:
holt_bm_gc_reclaimed_totalis the cumulative full-sweep plus exact-reclaim
counter, whileholt_bm_gc_last_full_sweep_deferred_countis a gauge for
the most recently completed full reachability sweep only.
Fixed
- Made snapshot copy-on-write ownership durable through persisted leases and
epochs, preserving snapshot-shared frames across checkpoints and reopen
recovery. - Fenced physical GC against readers, writers, checkpoints, and reopen
recovery. Dropping-tree and structural-reclaim work is now persisted so a
crash cannot leak or resurrect blobs. - Enforced child-before-parent checkpoint durability and bounded FIFO
reclamation for crash-safe structural cleanup. - Fixed hot-update churn reaching spillover with only reclaimable dead bytes or
tombstoned leaves by compacting locally before migration. - Checkpoint retries now recapture a current snapshot after stale-snapshot
errors instead of reusing stale state. - File-backed stores now persist a reduced manifest high-water mark and
truncate packed accelerator/data files after tail-slot reclamation. This
prevents long-running delete/compact workloads from staying pinned to their
historical slot high-water mark after the tail is durably free. - File-backed vacuum now compacts live high-water slots into lower reusable
holes before tail truncation, carryingread.idxandvalue.seg
accelerator slots with the authoritative blob slot. This reduces packed-file
high-water bloat after delete-heavy workloads instead of relying on future
writes to reuse middle holes. - After slot compaction, file-backed vacuum hole-punches any remaining reusable
middle slots on Linux, returning physical filesystem blocks without changing
live GUID mappings. - Background checkpointing now opportunistically auto-vacuums when
tail-reclaimable space crosses the configured threshold. - WAL group-commit flushing no longer issues duplicate fsyncs when a sync
target is visible before the corresponding committed ring records are
readable by the flusher. The concurrent durability regression now checks the
stable invariant (syncs <= appends) instead of assuming scheduler-dependent
batching.
Upgrade notes
- Forward upgrades from 0.7.3 retain the authoritative manifest v6, WAL v3,
blob-frame, DB catalog, and checkpoint-image formats. The newread.idxand
value.segfiles are rebuildable accelerators rather than recovery truth. - In-place downgrade from 0.8.2 to 0.7.3 is not supported. Version 0.8.2 can
write a WAL batch encoding that 0.7.3 does not recognize, and downgrading
would also restore the older snapshot-reclamation behavior fixed here. Use a
logical checkpoint image and a fresh directory if rollback is required.
Validation
cargo fmt --all --checkcargo fmt --manifest-path tools/soak/Cargo.toml --checkgit diff --checkcargo clippy --workspace --all-features --all-targets --locked -- -D warningscargo clippy --manifest-path tools/soak/Cargo.toml --locked -- -D warningscargo test --workspace --lib --tests --examples --lockedcargo test --workspace --doc --lockedRUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --lockedcargo package --locked- NoKV compatibility probe: 0.7.3 data-directory reopen, continued writes,
checkpoint/reopen, and 0.7.3 checkpoint-image restore under 0.8.2.
v0.8.1
Fixed
- Hardened corrupted on-disk root pointers on read-side entry paths. Lookup,
routed indexed reads, and blob-topology scans now returnNodeCorrupt
instead of tripping a debug assertion when a persisted root child pointer is
the null sentinel. - Added system fault regressions for manifest corruption, authoritative blob
root corruption, read-index corruption fallback, value-segment corruption
fallback, store-directory removal, permission denial, and injected ENOSPC
checkpoint write failure.
Changed
- Expanded fuzz coverage for metadata-shaped delimiters (
/,:,|,#,
@,\) and publicTree::is_prefix_empty. - Added a dedicated
read_index_modelfuzz target that exercises multi-blob
spillover, checkpoint, reopen, constrained buffer pools, indexed positive and
negative reads, prefix liveness, delimiter scans, stale read-index fallback,
and repeated checkpoint/reopen cycles. - Extended the soak harness to mix public prefix-empty checks and metadata
delimiter scans in both single-tree and DB modes. - Extended the Verus model with delimiter-tagged component-summary invariants
and prefix-liveness tri-state safety.
Validation
cargo fmt --all --checkcargo fmt --manifest-path tools/soak/Cargo.toml --checkgit diff --checkcargo clippy --workspace --all-features --all-targets --locked -- -D warningscargo clippy --manifest-path tools/soak/Cargo.toml --locked -- -D warningscargo test --workspace --lockedcargo +nightly fuzz run atomic_model -- -runs=64cargo +nightly fuzz run db_model -- -runs=48cargo +nightly fuzz run read_index_model -- -runs=8VERUS=/Users/guochengsong/.local/bin/verus ./verified/verify.shcargo run --manifest-path tools/soak/Cargo.toml --locked -- --mode normal --dir target/holt-soak-local-normal --reset --duration-secs 2 --keys 1000 --ops 4000 --threads 2 --buffer-pool 16 --wal-sync falsecargo run --manifest-path tools/soak/Cargo.toml --locked -- --mode db-normal --dir target/holt-soak-local-db --reset --duration-secs 2 --keys 1000 --ops 4000 --threads 2 --buffer-pool 16 --wal-sync false
v0.8.0
Added
- Added checkpoint-built read indexes and value segments as the named
acceleration layer for page-granular indexed reads.read.idxstores the
routable directory, filters, component summaries, crossings, and inline small
values;value.segstores larger value payloads referenced by the index.
Both files are rebuildable accelerators and never the source of truth. - Added read-index component summaries for delimiter/prefix rollups, allowing
somelist(..., delimiter="/")paths to emitCommonPrefixentries without
pinning every child blob.
Changed
- Breaking observability/API cleanup. Public stats, metrics, and benchmark
output now useread_index_*,read_page_*, andvalue_segment_*names
instead of the oldercold_*/ sidecar terminology. - Breaking accelerator file rename. File-backed stores now use
read.idx
andvalue.seginstead ofcold.idxandcold.val. These files are
advisory and can be regenerated fromblobs.dat; committed data, WAL, and
manifest recovery remain authoritative. - Renamed the internal
cold_readmodule toread_indexand aligned walker,
buffer-manager, blob-store, tests, README, and metrics terminology around the
indexed-read model.
Validation
cargo check --workspace --all-features --lockedcargo clippy --workspace --all-features --all-targets --locked -- -D warningscargo test --workspace --all-features --lockedcargo test --test wal_tree_integration --all-features --locked
v0.7.3
Fixed
- Fixed read-only
view/snapshotcaptures allocating persistent blob
slots. Snapshot roots are now ephemeral in-memory frames; repeated scans or
views no longer growblobs.datunless concurrent live writes actually fork
shared frames. - Added a regression test that repeatedly captures read views on a file-backed
tree and verifies checkpointing does not increase the backing store size.
v0.7.2
Fixed
- Fixed nightly DB/crash soak failures where stale cross-blob routes could reach
a delete-fenced blob.DBno longer runs GUID-only background auto-merge;
DB-wide merge stays rooted in live trees through explicit compaction. The
route cache is now restricted to root-child crossings, and walkers restart
from the root when they encounter a delete-fenced child instead of treating it
asNotFound. DB::viewnow uses the same fenced snapshot capture path asTree::view,
so multi-tree views cannot capture parent/child topology from mixed write
generations.- Merge eligibility now rejects snapshot-shared child blobs, preventing
maintenance from deleting a blob still referenced by a live snapshot.