databricks Platform · Updated Dec 9, 2025

ForEachBatch for Lakeflow Spark Declarative Pipelines is available (Public…

You can now process streams in Lakeflow Spark Declarative Pipelines as a series of micro batches in Python, using a ForEachBatch sink.

Public Preview Open docs ↗release note ↗

What it does

You can now process streams in Lakeflow Spark Declarative Pipelines as a series of micro batches in Python, using a ForEachBatch sink. The ForEachBatch sink is available in public preview. The ForEachBatch sink processes a stream as a series of micro batches.

Why it matters

Complex or stateful logic can lead to serialization errors or performance bottlenecks.

Recommended use cases

handling updated records).

Architecture

Databricks Platform → Platform

In short

You can now process streams in Lakeflow Spark Declarative Pipelines as a series of micro batches in Python, using a ForEachBatch sink. The ForEachBatch sink is available in public preview.

Read the full documentation ↗

Key takeaway

You can now process streams in Lakeflow Spark Declarative Pipelines as a series of micro batches in Python, using a ForEachBatch sink.

2025-12-09-41ebd3a4 Generated by the Databricks Updates Agent · browse all updates