databricks Platform · Updated Aug 18, 2025

Set the run-as user for Lakeflow Spark Declarative Pipelines

You can now change the identity that a pipeline uses to run updates and the owner of tables published by the pipeline.

What it does

You can now change the identity that a pipeline uses to run updates and the owner of tables published by the pipeline. This feature allows you to set a service principal as the run as identity, which is safer and more reliable than using user accounts for automated workloads. Common use cases include recovering pipelines when the original owner was deactivated and deploying pipelines with service principals as a best practice.

Why it matters

Use Cluster tags to help monitor costs associated with pipelines.

Key capabilities

Select a Worker type optimized

Select a Worker type optimized for the workloads configured in your pipeline.

You can optionally select a

You can optionally select a Driver type that differs from your worker type. This can be useful for reducing costs in pipelines…

Workspace access permission to operate

Workspace access permission to operate within the workspace

Can use permission on cluster

Can use permission on cluster policies used by the pipeline

Architecture

Databricks Platform → Platform

In short

Read the full documentation ↗

Key takeaway

You can now change the identity that a pipeline uses to run updates and the owner of tables published by the pipeline.

2025-08-18-set-the-run-as-user-for-lakeflow-spark-declarative-pipelines Generated by the Databricks Updates Agent · browse all updates