Operational tooling for DuckLake catalogs: visibility and safe cleanup for the inlined-data tables piling up after the 1.0 upgrade

dev tool real project •• multiple requests

Teams adopting DuckLake 1.0 on Postgres or MySQL catalogs are accumulating thousands of orphaned ducklake_inlined_data* tables (data inlining is on by default) with no clear way to inspect, vacuum, or compact the catalog. The built-in SQL primitives are low-level and behave inconsistently across catalog backends, and there is no day-2 operational tool (an Iceberg-style table-maintenance and health layer) for the format yet. The opportunity is a read-only catalog X-ray plus safe, dry-run maintenance, since the catalog lives in a database the user already controls.

builder note

Do not build another vacuum wrapper, DuckDB will fix its own primitives and obsolete it. The durable wedge is read-only visibility: the catalog is just tables in Postgres or MySQL, so show people the snapshot, inlined-table, and orphaned-file counts and the storage-versus-live ratio they cannot see today, with a safe dry-run before anything gets deleted.

landscape (2 existing solutions)

DuckLake 1.0 shipped mid-2026 and adoption is climbing, but its ecosystem has interfaces, libraries, and transforms and an empty operational-management column. Catalog maintenance is being patched reactively, issue by issue, inside DuckDB core.

DuckLake built-in maintenance functions (ducklake_delete_orphaned_files, ducklake_rewrite_data_files, snapshot expiry) Low-level SQL primitives, not an operational tool. No health report, no dry-run preview, no scheduling, and behavior differs by catalog backend (a maintainer could not reproduce inlined-table cleanup on a duckdb catalog while Postgres users pile up tens of thousands of tables).

Apache Iceberg table-maintenance ecosystem Iceberg has mature compaction and expiry tooling, but it assumes the Iceberg plus Spark or Trino stack. None of it targets DuckLake's lightweight SQL-catalog model, and DuckLake's whole appeal is avoiding that heavy stack.

sources (2)

other https://github.com/duckdb/ducklake/issues/1237 "the underlying ducklake_inlined_data* tables are not being removed" 2026-06-11

other https://github.com/duckdb/ducklake/issues/1219 "I would have expected the ducklake_inlined_data_*_* files to be removed ... but they are not" 2026-06-03

ducklakeduckdblakehousedata-infrastructurecatalog-maintenance