You can now serve custom and fine tuned LLMs on Databricks with a vLLM engine, including multimodal models and PEFT recipes that Foundation Model APIs does not support.
You can now serve custom and fine tuned LLMs on Databricks with a vLLM engine, including multimodal models and PEFT recipes that Foundation Model APIs does not support. Custom LLM Serving is in Beta. Beta This feature is in Beta.
Select a GPU based on your model's memory and performance needs.
You can now serve custom and fine tuned LLMs on Databricks with a vLLM engine, including multimodal models and PEFT recipes that Foundation Model APIs does not support. Custom LLM Serving is in Beta.
Read the full documentation ↗