Apache Arrow support for Zerobus Ingest in Lakeflow Connect is now generally available.
Apache Arrow support for Zerobus Ingest in Lakeflow Connect is now generally available. Arrow Flight ingestion lets you send Apache Arrow RecordBatch data directly to Zerobus Ingest instead of converting every row to JSON or Protocol Buffers (protobuf) first. It is a third record format option in Zerobus SDKs that support Arrow Flight, alongside JSON and protobuf, and it runs over the same gRPC connection.
Prefer this when the client is CPU constrained but still wants to reduce bytes on the wire.
Apache Arrow support for Zerobus Ingest in Lakeflow Connect is now generally available.
Read the full documentation ↗