databricks Platform · Updated Sep 16, 2025

Discover files in Auto Loader efficiently using file events without enrollment…

The cloudFiles.useManagedFileEvents option with Auto Loader is now in ungated Public Preview.

Public Preview Open docs ↗release note ↗

What it does

The cloudFiles.useManagedFileEvents option with Auto Loader is now in ungated Public Preview. This option allows you to discover files efficiently. For details, see the following: (Pre requisite) Enable file events for an external location File notification mode with and without file events enabled on external locations This page describes how to configure Auto Loader streams to use file notification mode to incrementally discover and ingest cloud data.

Why it matters

You can use file notifications to scale Auto Loader to ingest millions of files an hour.

Key capabilities

Databricks can set up subscriptions

Databricks can set up subscriptions and file events in your cloud storage account for you without requiring that you supply…

You have fewer IAM role

You have fewer IAM role policies to create in your cloud storage account.

Because you no longer need

Because you no longer need to create a queue for each Auto Loader stream, it's easier to avoid hitting the cloud provider…

Databricks automatically manages the tuning

Databricks automatically manages the tuning of resource requirements, so you don't need to tune parameters such as…

Architecture

Databricks Platform → Platform

In short

The cloudFiles.useManagedFileEvents option with Auto Loader is now in ungated Public Preview. This option allows you to discover files efficiently.

Read the full documentation ↗

Key takeaway

The cloudFiles.useManagedFileEvents option with Auto Loader is now in ungated Public Preview.

2025-09-16-0b9c0bfe Generated by the Databricks Updates Agent · browse all updates