---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 2f4762d03866d68c079a466866211806eb4894f33c139a5dfe014ef8472275f7
  pageDirectory: concepts
  sources:
    - how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md
  confidence: 0.9
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - dataops-on-databricks
    - DOD
    - Data types in Databricks
  citations:
    - file: how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md
title: DataOps on Databricks
description: Practice of applying CI/CD principles to data pipelines within the Databricks platform, ensuring reliable and secure data for ML models.
tags:
  - data-engineering
  - mlops
  - databricks
timestamp: "2026-06-19T19:06:40.429Z"
---

# DataOps on Databricks

**DataOps** is a set of practices and tools that apply the principles of DevOps and CI/CD to data pipelines, with the goal of making data management reliable, secure, repeatable, and automated. On the Databricks Data Intelligence Platform, DataOps ensures that the entire data lifecycle—from ingestion through transformation to model output—is managed on a single, unified platform for improved productivity and reproducibility. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

## Overview

Good [Machine Learning (ML)](/concepts/cicd-for-machine-learning.md) models depend on reliable data pipelines and infrastructure. DataOps on Databricks brings together techniques from MLOps, ModelOps, and DevOps to create a consistent and predictable workflow for data engineering and data science teams. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

The core principle of DataOps is that the entire data pipeline—from raw data ingestion to the outputs of a served [Machine Learning (ML)](/concepts/cicd-for-machine-learning.md) model—should be on a single platform and use the same toolset. This unified approach facilitates productivity, reproducibility, sharing, and troubleshooting. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

## Key Components and Tools

Databricks integrates several components to support DataOps workflows. The following table lists common DataOps tasks and the corresponding Databricks tools used to accomplish them:

| Task | Databricks Tool |
| :--- | :--- |
| Reliable data pipelines | [Lakeflow Jobs](/concepts/lakeflow-jobs.md) |
| Data quality monitoring | [Unity Catalog](/concepts/unity-catalog.md) |
| Data lineage and governance | [Unity Catalog](/concepts/unity-catalog.md) |
| Data cataloging | [Unity Catalog](/concepts/unity-catalog.md) |
| Data versioning, access, and sharing | [Unity Catalog](/concepts/unity-catalog.md) |
| Data pipeline monitoring | Lakeflow Monitoring |

*Task and tool data sourced from the reference document.* ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

### Reliable Data Pipelines
Databricks supports the automation and scheduling of data pipelines through [Lakeflow Jobs](/concepts/lakeflow-jobs.md), which can handle any workload type, from Extract, Transform, Load (ETL) to machine learning. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

### Data Quality and Governance
[Unity Catalog](/concepts/unity-catalog.md) provides centralized governance for all data and AI assets, including fine-grained access control, security policies, and data lineage. This allows teams to track data quality, schema changes, and distribution changes as part of the CI/CD workflow. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

## DataOps in the CI/CD for ML Lifecycle

DataOps is a critical element in the [Automated CI/CD pipeline for ML](/concepts/mlops-stacks.md) lifecycle. In the context of machine learning applications, CI/CD is important not only for code assets, but also for data pipelines, including both input data and the results generated by the model. ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

The following elements should be tracked in an automated CI/CD workflow as part of DataOps:

- **Training data**, including data quality, schema changes, and distribution changes.
- **Input data pipelines**.
- **Code** for training, validating, and serving the model.
- **Model predictions and performance**.

*All sourced from the reference document.* ^[how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md]

## Related Concepts
- DataOps
- [CI/CD for Machine Learning](/concepts/cicd-for-machine-learning.md)
- [Lakeflow Jobs](/concepts/lakeflow-jobs.md)
- [Unity Catalog](/concepts/unity-catalog.md)
- Lakeflow Monitoring
- MLOps on Databricks
- ModelOps
- DevOps
- [Automated CI/CD pipeline for ML](/concepts/mlops-stacks.md)
- Extract, Transform, Load (ETL)

## Sources
- how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md

# Citations

1. [how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws.md](/references/how-does-databricks-support-cicd-for-machine-learning-databricks-on-aws-48551477.md)
