databricks Platform · Updated Jun 26, 2026

Databricks Runtime 19 (Beta): June 26, 2026

What it does

Behavioral changes Environment variable availability restricted in standard access mode: In standard access mode, only a predefined set of environment variables is available to the Spark engine and init scripts. This set includes common configuration variables, such as proxy settings (HTTP PROXY), cloud credentials (AWS ACCESS KEY ID, AZURE CLIENT ID), and catalog variables (CATALOG). Other variables that you set on a cluster remain available to your user code, including UDFs, but are not available to the Spark engine or init scripts.

Why it matters

Reduces operational overhead and unlocks new capabilities for data teams on Databricks.

Limitations · remember

!Databricks recommends using the spark variable to interact with the SparkSession instance.

!The following sc functions are also not supported: emptyRDD, range, initbatchedserializer, parallelize, pickleFile, textFile, wholeTextFiles, binaryFiles, binaryRecords, sequenceFile, newAPIHadoopFile, newAPIHadoopRDD, hadoopFile, hadoopRDD, union, runJob, setSystemProperty, uiWebUrl, stop, setJobGroup, setLocalProperty, getConf. Setting certain Spark configuration properties is not supported. Creating or editing a cluster that sets a restricted property fails with an error. For the complete list, see Spark configuration limitations. When creating a DataFrame from local data using spark.createDataFrame, row sizes cannot exceed 128MB. RDD APIs are not supported. Spark Connect, which is used in more recent versions of Databricks Runtime, defers analysis and name resolution to execution time, which may change the behavior of your code. See Compare Spark Connect to Spark Classic.

!kafka.sasl.client.callback.handler.class

!kafka.sasl.login.callback.handler.class

!kafka.sasl.login.class

Architecture

Databricks Platform → Platform

In short

Read the full documentation ↗

Key takeaway

2026-06-26-databricks-runtime-19-beta-june-26-2026 Generated by the Databricks Updates Agent · browse all updates