Enhanced support for semi-structured data in compliance workspaces.
The `VARIANT` data type allows for the storage and querying of semi-structured data from various sources, including streaming services and REST APIs. It is supported in all Apache Iceberg v3 tables and can be enabled in Delta Lake tables with specific commands. Users must operate on Databricks Runtime 15.4 LTS or higher to utilize this feature effectively.
This update enhances the ability of organizations to ingest and analyze semi-structured data, which is increasingly vital for real-time analytics and data-driven decision-making. By making `VARIANT` available by default in compliance-focused workspaces, it streamlines the process for data engineers and analysts, improving productivity and enabling more robust data governance.