![[project]holt: an experimental Rust metadata index built around persistent ART blobs, WAL, and checkpointing](https://external-preview.redd.it/-axCeKnkkZOnwMWagYd7WRITLYeeK4JLZZMP3ArLzZ0.png?width=1080&crop=smart&auto=webp&s=3b07a721ba575ec504454a5c079ce3c734a74050)
[project]holt: an experimental Rust metadata index built around persistent ART blobs, WAL, and checkpointing
Hi r/rust,
I’m the author of `holt`, an experimental storage-engine project written in Rust.
The goal is to explore a metadata index for object-store / filesystem-like namespaces, not a general SQL database. The current design stores keys in an ART-like tree where each durable blob is a fixed-size 512 KiB page. When a blob fills up, a subtree can spill into another blob through a `BlobNode`. Persistence is handled through a WAL, dirty blob tracking, and checkpointing.
Repo: https://github.com/feichai0017/holt
What exists today:
- fixed-size blob/page layout
- ART-style nodes: Leaf, Prefix, Node4/16/48/256, BlobNode
- persistent backend over a packed `blobs.dat` file
- manifest mapping BlobGuid -> physical slot
- WAL replay for logical operations
- write-back buffer manager
- background checkpoint prototype
- range iterator and metadata stats
What I’m trying to get feedback on:
Whether the WAL / checkpoint / dirty-blob protocol is the right direction.
How to evolve spillover into a scalable page split / routing model.
Whether this layout makes sense for object storage or filesystem metadata, where values are small metadata records or external data references.
What failure cases you would test before trusting the design further.
This is not production-ready. I’m mainly looking for storage-engine and Rust systems feedback, especially around crash recovery and large-scale layout.