databricks Platform · Updated Jul 28, 2026

Python UDTFs in Unity Catalog

User-defined table functions are now generally available for enhanced data processing.

What it does

Python user-defined table functions (UDTFs) in Unity Catalog allow users to register functions that return tables instead of single values, enabling complex data transformations and integrations directly within SQL queries. These functions can be invoked in the `FROM` clause of SQL statements and can accept both scalar and table arguments. UDTFs are supported on various compute types, including serverless and classic compute environments.

Why it matters

This feature enhances data processing capabilities by allowing users to create more complex and flexible SQL queries, which can lead to improved data insights and operational efficiency. Organizations can leverage UDTFs to integrate external data sources, perform advanced transformations, and enrich datasets, ultimately driving better decision-making and reducing time-to-insight.

Key capabilities

Table Function Registration

Dynamic Output Schema

Create polymorphic UDTFs that determine their output schema at query time.

Stateful Operations

Implement stateful processing across rows using the `eval()` and `terminate()` methods.

Environment Isolation

Control execution environments with strict isolation for UDTFs that modify global state.

Prerequisites

Databricks Runtime 17.1 or above for classic compute.

Serverless compute support for UDTFs.

Recommended use cases

Transforming complex data structures into tabular formats.Integrating external APIs into SQL workflows.Implementing custom data generation logic.Processing data requiring stateful operations.

Limitations · remember

!Polymorphic UDTFs require Databricks Runtime 18.1 and above.

!Strict isolation may increase resource usage.

Architecture

Unity Catalog → SQL Warehouse → Serverless Compute

Get started

Create UDTF

Use SQL DDL to create a governed UDTF in Unity Catalog.

Invoke UDTF

Call the UDTF in a SQL statement's FROM clause.

Key takeaway

Python UDTFs in Unity Catalog empower users to perform complex data transformations directly within SQL, enhancing data processing capabilities.

2026-07-28-python-udtfs-in-unity-catalog-are-now-generally-available Generated by the Databricks Updates Agent · browse all updates