← All updates Databricks Update One-Pager Navy version
databricks Platform · Updated Jul 22, 2026

Variant and Variant Shredding

Flexible data type for semi-structured data now generally available.

GA Open docs ↗release note ↗
What it does

Variant is a flexible data type designed for semi-structured data, enabling efficient ingestion from sources like Kinesis, Event Hub, REST APIs, and schemaless databases. With the introduction of Variant shredding, which is automatically enabled in Databricks Runtime 17.3 and above, users can achieve structured-like query performance. For optimal performance, Databricks recommends using Runtime 18.1 or higher, which enhances variant statistics collection and data skipping capabilities.

Why it matters

The availability of Variant and Variant shredding significantly improves the performance and efficiency of querying semi-structured data, which is crucial for organizations dealing with diverse data sources. This enhancement allows data teams to streamline their workflows, reduce query times, and ultimately lower operational costs while ensuring compliance with data governance standards.

Key capabilities
Flexible Data Ingestion
Supports ingestion from various sources like streaming services and REST APIs.
Automatic Shredding
Variant shredding is automatically enabled for new tables with VARIANT columns.
Enhanced Query Performance
Delivers structured-like query performance through optimized statistics collection.
Compliance Ready
Variant is compatible with workspaces using the compliance security profile.
Prerequisites
Databricks Runtime 17.3 or above for Variant shredding.
Databricks Runtime 18.1 or above for optimal performance.
Recommended use cases
Ingesting semi-structured data from Kinesis or Event Hub.Querying data from REST APIs efficiently.Processing data from schemaless databases.
Architecture
Data Ingestion Layer → Variant Data Type → Query Optimization Layer → Compliance Security Profile
In short

Variant and Variant shredding are now available for better handling of semi-structured data.

Read the full documentation ↗
Key takeaway
The introduction of Variant and Variant shredding enhances the ability to efficiently manage and query semi-structured data.
2026-07-22-variant-and-variant-shredding-are-now-generally-available Generated by the Databricks Updates Agent · browse all updates