Lakewright solutions / brief

Migrate a Parquet data swamp to governed Apache Iceberg tables

A bucket of accumulated Parquet files becomes a set of governed Iceberg tables — schemas read from footers, duplicates skipped by content digest, existing Delta/Iceberg/Hudi tables detected and left alone.

How does it handle a mixed bucket?

Parquet footers give exact schemas; files clustering to the same schema become one feed. Directories that are already table formats (_delta_log, Iceberg metadata, .hoodie) are recognized as existing tables and reported — never re-ingested file-by-file.

The steps

lakewright scan  s3://data-lake-old        # inventory: feeds + existing tables
lakewright approve plans/events_archive.plan.yaml
lakewright load  plans/events_archive.plan.yaml --rest $CATALOG

Avro lands the same way. Snowflake, Databricks, Trino, and DuckDB query the result directly.

FAQ