New open-source repo for standardization of messy data into geoparquet
Been working on an open-source geospatial ETL prototype called Dymium focused on standardizing fragmented geological datasets into ML-ready GeoParquet outputs.
Current pipeline handles:
- MRDS ingestion and normalization
- geological PDF extraction
- cross-source dataset fusion
- spatial geology enrichment
- GeoParquet export
- lightweight Streamlit visualization
My main motivation was seeing how much mineral/geological data is still trapped across inconsistent schemas, PDFs, shapefiles, and legacy formats.
Still very early-stage and intentionally scoped around the data-standardization layer rather than full modeling. README includes current limitations, uncertainty handling examples, and demo outputs.
I need feedback from GIS/geospatial/data engineering people — especially around:
- schema normalization approaches
- GeoParquet workflows
- geology layer enrichment
- ingestion validation
- interoperability issues across jurisdictions
u/mmscoin — 3 days ago