← All updates Databricks Update One-Pager Light version
databricks Platform · Updated Jul 9, 2026

Protobuf Tensor Input

Now supports serialized KServe v2 requests for custom model endpoints.

Public Preview Open docs ↗release note ↗
What it does

This update allows users to send requests to custom model serving endpoints using a serialized KServe v2 `ModelInferRequest` with `Content-Type: application/x-protobuf`, instead of the traditional JSON format. This method enhances performance by reducing the overhead of parsing numeric tensors from JSON, particularly beneficial for large payloads. The feature is available only for endpoints deployed after July 9, 2026.

Why it matters

By enabling Protobuf tensor input, organizations can achieve faster model inference times, which is crucial for applications requiring real-time predictions. This enhancement can lead to improved user experiences and operational efficiency, especially in high-demand environments such as AI and machine learning applications.

Key capabilities
Faster Inference
Reduces the overhead of parsing numeric tensors, improving response times.
Supports KServe v2
Utilizes the KServe v2 `ModelInferRequest` format for input.
Tensor Data Types
Supports various tensor data types including FP32, INT32, and BOOL.
Endpoint Compatibility
Maintains existing endpoint URL and authentication methods.
Prerequisites
A model serving endpoint must be created.
MLflow 2.9 or above is required for the MLflow Deployment SDK.
A valid Databricks API token is needed for requests.
Recommended use cases
Real-time predictions in AI applications.High-performance model serving for large datasets.Integration with existing KServe-based workflows.
Limitations · remember
!Only available for endpoints deployed after July 9, 2026.
!Route optimization endpoints require OAuth tokens and have different invocation URLs.
Architecture
Model Serving → Custom Model Endpoints → KServe Integration
In short

You can now send requests to custom model endpoints using a faster Protobuf format instead of JSON.

Read the full documentation ↗
Key takeaway
The introduction of Protobuf tensor input streamlines the request process for custom model serving, enhancing performance and efficiency.
2026-07-09-protobuf-tensor-input-for-custom-model-serving-endpoints-public-preview Generated by the Databricks Updates Agent · browse all updates