---
title: Deploy models using Model Serving | Databricks on AWS
source: https://docs.databricks.com/aws/en/machine-learning/model-serving/
ingestedAt: "2026-06-18T08:11:40.108Z"
---

This article describes Model Serving, the Databricks solution for deploying AI and ML models for real-time serving and batch inference.

## What is Model Serving?[​](#what-is-model-serving "Direct link to What is Model Serving?")

Model Serving provides a unified interface to deploy, govern, and query AI models for real-time and batch inference. Each model you serve is available as a REST API that you can integrate into your web or client application.

Model Serving provides a highly available and low-latency service for deploying models. The service automatically scales up or down to meet demand changes, saving infrastructure costs while optimizing latency performance. This functionality uses [serverless compute](https://docs.databricks.com/aws/en/getting-started/high-level-architecture#serverless). See the [Model Serving pricing page](https://www.databricks.com/product/pricing/model-serving) for more details.

Model Serving offers a unified REST API and MLflow Deployment API for CRUD and querying tasks. In addition, it provides a single UI to manage all your models and their respective serving endpoints. You can also access models directly from SQL using [AI Functions](https://docs.databricks.com/aws/en/large-language-models/ai-functions) for easy integration into analytics workflows.

AI Functions and Model Serving are tightly integrated for batch inference scenarios. You can use any of the task-specific AI Functions or `ai-query` in your batch inference pipelines. If you choose to use a pre-provisioned model that is hosted and managed by Databricks, you don't need to configure a model serving endpoint yourself.

See the following guides to get started:

*   For performing batch inference, see [Enrich data using AI Functions](https://docs.databricks.com/aws/en/large-language-models/ai-functions).
*   For an introductory tutorial on how to serve custom models on Databricks for real-time inference, see [Tutorial: Deploy and query a custom model](https://docs.databricks.com/aws/en/machine-learning/model-serving/model-serving-intro).
*   For a getting started tutorial on how to query a foundation model on Databricks for real-time inference, see [Get started querying LLMs on Databricks](https://docs.databricks.com/aws/en/large-language-models/llm-serving-intro).

## Models you can deploy[​](#models-you-can-deploy "Direct link to Models you can deploy")

Model serving supports real time and batch inference for the following model types:

*   [Custom models](https://docs.databricks.com/aws/en/machine-learning/model-serving/custom-models). These are Python models packaged in the MLflow format. They can be registered either in Unity Catalog or in the workspace model registry. Examples include scikit-learn, XGBoost, PyTorch, and Hugging Face transformer models.
    *   Agent serving is supported as a custom model. See [Deploy an agent for generative AI applications (Model Serving)](https://docs.databricks.com/aws/en/generative-ai/agent-framework/deploy-agent)
*   [Foundation models](https://docs.databricks.com/aws/en/machine-learning/model-serving/foundation-model-overview).
    *   **Databricks-hosted foundation models** like Meta Llama. These models are available using [Foundation Model APIs](https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/). These models are curated foundation model architectures that support optimized inference. Base models, like Meta-Llama-3.3-70B-Instruct, GTE-Large, and Mistral-7B are available for immediate use with **pay-per-token** pricing, and workloads that require performance guarantees and fine-tuned model variants can be deployed with **provisioned throughput**.
    *   **Foundation models hosted outside of Databricks** like GPT-4 from OpenAI. These models are accessible using [External models](https://docs.databricks.com/aws/en/generative-ai/external-models/). The endpoints that serve these models can be centrally governed from Databricks, so you can streamline the use and management of various LLM providers, such as OpenAI and Anthropic, within your organization.

note

You can interact with supported large language models using the [AI Playground](https://docs.databricks.com/aws/en/large-language-models/ai-playground). The AI Playground is a chat-like environment where you can test, prompt, and compare LLMs. This functionality is available in your Databricks workspace.

## Why use Model Serving?[​](#why-use-model-serving "Direct link to Why use Model Serving?")

*   **Deploy and query any models**: Model Serving provides a unified interface that so you can manage all models in one location and query them with a single API, regardless of whether they are hosted on Databricks or externally. This approach simplifies the process of experimenting with, customizing, and deploying models in production across various clouds and providers.
*   **Securely customize models with your private data**: Built on a Data Intelligence Platform, Model Serving simplifies the integration of features and embeddings into models through native integration with the [Databricks Feature Store](https://docs.databricks.com/aws/en/machine-learning/feature-store/) and [AI Search](https://docs.databricks.com/aws/en/ai-search/ai-search). For even more improved accuracy and contextual understanding, models can be fine-tuned with proprietary data and deployed effortlessly on Model Serving.
*   **Govern and monitor models**: The Serving UI allows you to centrally manage all model endpoints in one place, including those that are externally hosted. You can manage permissions, track and set usage limits and monitor the quality of all types of models using [AI Gateway](https://docs.databricks.com/aws/en/ai-gateway/). This enables you to democratize access to SaaS and open LLMs within your organization while ensuring appropriate guardrails are in place.
*   **Reduce cost with optimized inference and fast scaling**: Databricks has implemented a range of optimizations to ensure you get the best throughput and latency for large models. The endpoints automatically scale up or down to meet demand changes, saving infrastructure costs while optimizing latency performance. [Monitor model serving costs](https://docs.databricks.com/aws/en/admin/system-tables/model-serving-cost).
    *   For workloads that are latency sensitive or involve a high number of queries per second, see [Optimize Model Serving endpoints for production](https://docs.databricks.com/aws/en/machine-learning/model-serving/production-optimization) for comprehensive optimization strategies. Reach out to your Databricks account team to ensure your workspace is enabled for high scalability.

*   **Bring reliability and security to Model Serving**: Model Serving is designed for high-availability, low-latency production use and can support over 25K queries per second with an overhead latency of less than 50 ms. The serving workloads are protected by multiple layers of security, ensuring a secure and reliable environment for even the most sensitive tasks. You can control network access to Model Serving endpoints by configuring network policies. See [Manage network policies for serverless egress control](https://docs.databricks.com/aws/en/security/network/serverless-network-security/manage-network-policies).

note

Model Serving does not provide security patches to existing model images because of the risk of destabilization to production deployments. A new model image created from a new model version will contain the latest patches. Reach out to your Databricks account team for more information.

## Requirements[​](#requirements "Direct link to requirements")

*   Registered model in [Unity Catalog](https://docs.databricks.com/aws/en/data-governance/unity-catalog/) or the [Workspace Model Registry](https://docs.databricks.com/aws/en/machine-learning/manage-model-lifecycle/workspace-model-registry).
*   Permissions on the registered models as described in [Serving endpoint ACLs](https://docs.databricks.com/aws/en/security/auth/access-control/#serving-endpoints).
*   MLflow 1.29 or higher.
*   Workspace entitlements configured. See [Manage entitlements](https://docs.databricks.com/aws/en/security/auth/entitlements).

## Enable Model Serving for your workspace[​](#enable-model-serving-for-your-workspace "Direct link to enable-model-serving-for-your-workspace")

To use Model Serving, your account admin must read and accept the terms and conditions for enabling serverless compute in the account console.

note

If your account was created after March 28, 2022, serverless compute is enabled by default for your workspaces.

If you are not an account admin, you cannot perform these steps. Contact an account admin if your workspace needs access to serverless compute.

1.  As an account admin, go to the [feature enablement tab of the account console settings page](https://accounts.cloud.databricks.com/settings/feature-enablement).
2.  A banner at the top of the page prompts you to accept the additional terms. Once you read the terms, click **Accept**. If you do not see the banner asking you to accept the terms, this step has been completed already.

After you've accepted the terms, your account is enabled for serverless.

No additional steps are required to enable Model Serving in your workspace.

## Limitations and region availability[​](#limitations-and-region-availability "Direct link to Limitations and region availability")

Model Serving imposes default limits to ensure reliable performance. See [Model Serving limits and regions](https://docs.databricks.com/aws/en/machine-learning/model-serving/model-serving-limits). If you have feedback on these limits or an endpoint in an unsupported region, reach out to your Databricks account team.

## Data protection in Model Serving[​](#-data-protection-in-model-serving "Direct link to -data-protection-in-model-serving")

Databricks takes data security seriously. Databricks understands the importance of the data you analyze using Model Serving, and implements the following security controls to protect your data.

*   Every customer request to Model Serving is logically isolated, authenticated, and authorized.
*   Model Serving encrypts all data at rest (AES-256) and in transit (TLS 1.2+).

For all paid accounts, Model Serving does not use user inputs submitted to the service or outputs from the service to train any models or improve any Databricks services.

For all Model Serving workloads, Databricks retains container build logs for up to thirty (30) days and metrics data for up to fourteen (14) days.

For Databricks Foundation Model APIs, as part of providing the service, Databricks may temporarily process and store inputs and outputs for the purposes of preventing, detecting, and mitigating abuse or harmful uses. Your inputs and outputs are isolated from those of other customers, stored in the same region as your workspace for up to thirty (30) days, and only accessible for detecting and responding to security or abuse concerns.

Our partner model providers may retain data for safety purposes. This retention relies on automated scanning prior to any limited human review. Models with safety retention requirements are noted on [Databricks-hosted foundation models available in Foundation Model APIs](https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models).

Foundation Model APIs is a [Databricks Designated Service](https://docs.databricks.com/aws/en/resources/designated-services), meaning it adheres to data residency boundaries as implemented by [Databricks Geos](https://docs.databricks.com/aws/en/resources/databricks-geos).

## Additional resources[​](#additional-resources "Direct link to Additional resources")

*   [Get started querying LLMs on Databricks](https://docs.databricks.com/aws/en/large-language-models/llm-serving-intro).
*   [Tutorial: Deploy and query a custom model](https://docs.databricks.com/aws/en/machine-learning/model-serving/model-serving-intro)
*   [Tutorial: Create external model endpoints to query OpenAI models](https://docs.databricks.com/aws/en/generative-ai/tutorials/external-models-tutorial)
*   [Build agents on Databricks](https://docs.databricks.com/aws/en/generative-ai/agent-framework/build-agents)
*   [Enrich data using AI Functions](https://docs.databricks.com/aws/en/large-language-models/ai-functions)
*   [Migrate to Model Serving](https://docs.databricks.com/aws/en/machine-learning/model-serving/migrate-model-serving)
