Which table format to choose on Snowflake and Databricks?

I am proposing a simple guide to choose your table format on Snowflake and Databricks, only keeping the customers’ best interest in mind.

There is support for Apache Hudi on Snowflake and Apache Icerberg on Databricks but given the recent developments and how the opensource community and relevant companies are positioning themselves, you should try to avoid bad combinations.

If you are already on Databricks and need a datawarehouse, then you can consider Databricks SQL ( https://www.databricks.com/product/databricks-sql ).

With all the three table formats: Delta, Icerberg, and Hudi currently supporting similar set of features and also closing the gaps for missing features, I would think they will all converge and look very similar in future, although there might be differences in how a given feature is implemented, since they manage their metadata differently.

Also, if you want to be table format agnostic or want to support all the table formats, you can also consider https://onetable.dev/ , which lets you interoperate between all the three table formats.

Below is my recommendation for the table format.

#iceberg #hudi #delta #tableformat #dataengineering #datawarehouse

#data #dataanalytics