databricks Platform · Updated Jul 7, 2026

REPLACE WHERE Flows

Now generally available for efficient data processing.

What it does

REPLACE WHERE flows allow for targeted updates of a table by replacing only the rows that match a specified predicate, without the need to reprocess the entire table history. This mechanism supports late-arriving data, schema evolution, and selective reprocessing, making it ideal for incremental batch processing. The flows can be defined using SQL or Python, and they can handle complex scenarios beyond standard materialized views.

Why it matters

This feature significantly enhances data processing efficiency by reducing the computational overhead associated with full table refreshes. Businesses can achieve faster data updates, lower operational costs, and improved data accuracy, ultimately leading to better decision-making and responsiveness to changing data conditions.

Key capabilities

Incremental Batch Processing

Process new rows in batches without managing streaming complexities.

Selective Reprocessing

Recompute only rows matching a predicate while leaving others untouched.

Schema Evolution Support

Handle schema changes without full recomputation of historical data.

Backfill Historical Data

Easily backfill historical data using predicate overrides or DML statements.

Prerequisites

Unity Catalog is recommended.

Serverless compute is required for incremental refresh.

Recommended use cases

Updating sales data with new transactions.Maintaining historical aggregates from a limited-retention source.Correcting data for specific historical periods.

Limitations · remember

!Only one REPLACE WHERE flow is allowed per target table.

!The target table must be created within the pipeline.

!Expectations are not supported on tables targeted by REPLACE WHERE flows.

Architecture

Source Data → REPLACE WHERE Flow → Target Table → Incremental Refresh

Get started

Define Flow

Create a REPLACE WHERE flow using SQL or Python syntax.

Set Predicate

Specify the predicate for rows to be replaced.

Execute Flow

Run the flow to update the target table with new data.

Backfill Data

Use predicate overrides or DML for historical data corrections.

Monitor Performance

Check logs to ensure incremental refresh is functioning as expected.

Key takeaway

REPLACE WHERE flows streamline data updates by allowing selective row replacement, enhancing efficiency and reducing costs.

2026-07-07-replace-where-flows-are-now-generally-available Generated by the Databricks Updates Agent · browse all updates