ndatafusion: linear algebra and ML for DataFusion, powered by nabled
Hello r/rust! I just released ndatafusion 🪐
The goal is to make Apache DataFusion a much more natural place for linear algebra and ML-style workloads.
ndatafusion brings numerical capabilities into DataFusion through explicit Arrow/DataFusion contracts, powered under the hood by nabled, my other Rust crate for linalg/ML workloads.
I built it because I wanted a cleaner answer to a pretty basic problem: if you’re already using Arrow and DataFusion, you shouldn’t have to leave that ecosystem the moment you need vector or matrix-oriented computation.
As far as I know, there still isn’t much in the Rust/DataFusion ecosystem aimed directly at this layer, which is part of why I wanted to get it out.
Links:
- Crates.io: https://crates.io/crates/ndatafusion
- Docs.rs: https://docs.rs/ndatafusion
- GitHub: https://github.com/georgeleepatterson/ndatafusion
Would especially appreciate feedback from people working on Rust query engines, Arrow-native systems, SQL extensions, or ML/data infra.