databricks Platform · Updated Jul 23, 2026

C++ SDK for Zerobus Ingest Connector

Now in Beta, enabling high-throughput data ingestion.

What it does

The C++ SDK for the Zerobus Ingest connector allows users to ingest records directly into Unity Catalog Delta tables using a high-throughput gRPC client. It supports multiple data formats including JSON and Protocol Buffers, catering to various ingestion needs. The SDK is designed for high-volume applications, leveraging persistent connections to optimize performance. Additionally, it integrates seamlessly with existing Databricks infrastructure, facilitating efficient data workflows.

Why it matters

This SDK significantly enhances data ingestion capabilities for organizations, allowing them to handle large volumes of data efficiently and in real-time. By supporting multiple data formats and high-throughput ingestion, businesses can improve their data processing speed and reduce latency, leading to better decision-making and operational efficiency. This is particularly beneficial for industries reliant on real-time data analytics and monitoring.

Key capabilities

High-Throughput Ingestion

Utilizes gRPC for persistent connections, optimizing data ingestion speeds.

Multiple Data Formats

Supports JSON, Protocol Buffers, and Apache Arrow Flight for flexible data handling.

Integration with Unity Catalog

Directly ingests records into Unity Catalog Delta tables for streamlined data management.

OpenTelemetry Support

Ingests OpenTelemetry data without custom libraries, simplifying monitoring setups.

Prerequisites

C++ development environment set up.

Access to Databricks workspace with Zerobus Ingest connector enabled.

Knowledge of gRPC and data formats like JSON and Protocol Buffers.

Recommended use cases

Real-time data ingestion from IoT devices.High-volume data processing for analytics applications.Integration of telemetry data into data lakes.Batch processing of data using Apache Arrow Flight.

Limitations · remember

!Currently in Beta, may have limited support and features.

!Concurrency quotas apply due to persistent connections in gRPC.

!Not all features may be available in all regions.

Architecture

Lakeflow Connect → Zerobus Ingest Connector → gRPC Client → Unity Catalog Delta Tables

Get started

Set Up Environment

Ensure a C++ development environment is ready for SDK integration.

Obtain Workspace URL

Get your Databricks workspace URL and Zerobus Ingest endpoint.

Create Target Table

Define the Delta table schema in Unity Catalog for data ingestion.

Implement SDK Client

Use the C++ SDK to write a client for data ingestion.

Test Ingestion

Run tests to validate data ingestion into the target Delta table.

Key takeaway

The C++ SDK for the Zerobus Ingest connector empowers organizations to efficiently ingest large volumes of data into Databricks, enhancing real-time analytics capabilities.

2026-07-23-c-sdk-for-the-zerobus-ingest-connector-is-now-available-in-beta Generated by the Databricks Updates Agent · browse all updates