I've been working on a 4D voxel editor for a few years (see https://www.youtube.com/shorts/l2-vFT5cKHE) It's kind of like MagicaVoxel for 4D. Right now, the 4D voxels, AKA 'Hoxels', are stored in a flat grid. It works well enough for demonstration purposes, but I'm quickly running into GPU memory limitations. For example, a relatively small 64^3 grid would be a comfortable 262,144 voxels. However, a 64^4 grid takes up a staggering 16,777,216 hoxels.
I've been researching a bunch of different data structures, looking for something that can scale to 4D and has reasonable memory savings and ray marching speed. I understand VDB, Brickmaps, 64-trees, and SVOs from a high level. Right now I’m leaning toward Brickmaps. I like the idea of having a shallow, fixed-depth tree. I think I understand the memory layout. What I'm struggling with is how to design a system so that the voxel data can be edited quickly and copied to GPU memory.
Some other considerations are:
I need to support undo/redo functionality
I need to save the grid to disk fairly frequently
I'd like to have the entire model in GPU memory all the time
I don't need obscene sizes. If I could support 1024^4 grids, I would be happy. Right now, with flat grids, I’m maxing out around 128^4.
What is a high-level strategy for managing one of these data structures when edits are made? For example, what if I place a hoxel in a region that requires new nodes all the way up the tree? What if I delete the last hoxel in a branch and I can delete nodes all the way up the tree? Is there a straightforward way to do this or am I going to be writing an entire memory management system?