← All updates Databricks Update One-Pager Light version
databricks Platform · Updated Aug 14, 2025

Spark Declarative Pipelines on Lakeflow

New ETL pipeline creation template in workspace bundles.

What it does

The Spark Declarative Pipelines on Lakeflow template allows users to create ETL pipelines within bundles directly in the Databricks workspace. Users can select this template when creating a new bundle, which simplifies the process of defining data transformations and workflows. The template supports configurations such as default catalogs and schemas, and it integrates with the Lakeflow Pipelines Editor for enhanced functionality.

Why it matters

This update streamlines the ETL pipeline creation process, enabling data engineers to quickly set up and manage data workflows without extensive coding. By leveraging templates, organizations can reduce development time and improve consistency across projects, leading to enhanced productivity and faster time-to-insight.

Key capabilities
ETL Pipeline Creation
Users can create ETL pipelines using a predefined template in the workspace.
Customizable Configurations
The template allows for configuration of catalogs, schemas, and programming languages.
Integration with Lakeflow
Supports the Lakeflow Pipelines Editor for enhanced pipeline management.
Direct Bundle Management
Bundles can be created and modified directly within the Databricks workspace.
Prerequisites
Lakeflow Pipelines Editor must be enabled in the workspace.
Recommended use cases
Rapid development of ETL workflows.Standardization of data transformation processes.Collaboration among data engineers using shared templates.
Architecture
Databricks Workspace → Declarative Automation Bundles → Lakeflow Pipelines Editor → ETL Pipeline Management
Get started
1
Navigate to Git Folder
Go to the Git folder where you want to create your bundle.
2
Create a Bundle
Click the Create button and select Bundle.
3
Select Template
Choose the Lakeflow Spark Declarative Pipelines template.
4
Configure Bundle
Set the required configurations for the pipeline.
5
Create and Deploy
Finalize the setup by clicking Create and deploy.
Key takeaway
The new Spark Declarative Pipelines template simplifies ETL pipeline creation, enhancing productivity and consistency in data workflows.
2025-08-14-21a0396e Generated by the Databricks Updates Agent · browse all updates