Centralized governance for LLM endpoints across workspaces.
This feature allows users to govern Databricks-hosted LLMs through model services in Unity Catalog, enabling centralized management of LLM endpoints. Model services serve as securable objects that define, share, and govern access to LLMs across multiple workspaces. Users can create and manage these services using the Unity AI Gateway UI, Catalog Explorer, or REST API, and can query them from both inside and outside Databricks.
By centralizing the governance of LLMs, organizations can reduce duplication of effort and streamline access management across workspaces, enhancing productivity and governance. This feature allows for better tracking of usage and costs associated with LLMs, which is crucial for budgeting and resource allocation.