You can use Python custom data sources and sinks in your pipeline definitions in Lakeflow Spark Declarative Pipelines.
You can use Python custom data sources and sinks in your pipeline definitions in Lakeflow Spark Declarative Pipelines. For information about Python custom data sources, see the following: Load data from a Python custom data source. PySpark custom data sources.
Reduces operational overhead and unlocks new capabilities for data teams on Databricks.
You can use Python custom data sources and sinks in your pipeline definitions in Lakeflow Spark Declarative Pipelines. For information about Python custom data sources, see the following: Load data from a Python custom data source.
Read the full documentation ↗