"DuckLake 1.0 ships a stable specification, a feature-rich and fast reference implementation (the DuckDB ducklake extension), as well as a roadmap for future development."
"Data inlining is one of the flagship features of DuckLake. It basically enables performing small insert, delete and update operations in the catalog database, avoiding the proliferation of 'the small file problem'."
DuckLake 1.0 is a new data lake format that stores table metadata in a SQL database, addressing issues with file-based metadata in existing lake formats. This release includes features like catalog-stored updates, improved sorting, and partitioning options. DuckLake aims to resolve complexities and slow operations associated with file-based metadata. Key features include data inlining for small updates, sorted tables for faster queries, and enhanced support for geometry data types. The release is production-ready and ensures backward compatibility.
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]