---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: f1403b9c07cbafb377042618ef68de9cbfd5eaae7ad550a729638d3aaf316855
  pageDirectory: concepts
  sources:
    - classification-with-automl-databricks-on-aws.md
  confidence: 0.9
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - automl-experiment-monitoring-and-notebooks
    - Notebooks and AutoML Experiment Monitoring
    - AEMAN
  citations:
    - file: classification-with-automl-databricks-on-aws.md
title: AutoML Experiment Monitoring and Notebooks
description: Tools for monitoring AutoML experiment progress, viewing dataset warnings (e.g., unsupported column types, high cardinality), accessing data exploration notebooks, and reviewing trial run notebooks stored as MLflow artifacts.
tags:
  - monitoring
  - automl
  - notebooks
  - databricks
timestamp: "2026-06-18T10:57:16.489Z"
---

# AutoML Experiment Monitoring and Notebooks

**AutoML Experiment Monitoring and Notebooks** refers to the set of tools and practices for tracking, inspecting, and reusing the artifacts generated by an AutoML experiment in Databricks. When you run an AutoML classification, regression, or forecasting experiment, the system creates a parent [MLflow Experiment](/concepts/mlflow-experiment.md) containing all trial runs, a data exploration notebook, and per-trial source code notebooks. These artifacts allow you to monitor progress, diagnose issues, and later refine or deploy the best model. ^[classification-with-automl-databricks-on-aws.md]

## Monitoring Experiment Progress

After starting an AutoML experiment from the UI, the training page displays a runs table that you can refresh to see the latest status. From this page you can:

- Stop the experiment at any time.
- Open the data exploration notebook.
- Monitor individual trial runs.
- Navigate to the run page for any completed trial.

In Databricks Runtime 10.1 ML and above, AutoML also surfaces **warnings** about potential issues with the dataset, such as unsupported column types or high-cardinality columns. These warnings are shown on a **Warnings** tab on the training page and remain available on the experiment page after the experiment finishes. ^[classification-with-automl-databricks-on-aws.md]

## Viewing Results

When the experiment completes, the results page offers several options:

- **Register and deploy** the best model (based on the primary metric) to [Unity Catalog](/concepts/unity-catalog.md) or the [Model Registry](/concepts/mlflow-model-registry.md).
- **View notebook for best model** – opens the notebook that produced the best trial; you can review and edit this notebook to refine the solution.
- **View data exploration notebook** – opens the automatically generated exploratory analysis notebook.
- **Search, filter, and sort** the runs in the table.
- **Inspect any trial run** – clicking on a run’s **Models** column or **Start Time** navigates to the [MLflow Run](/concepts/mlflow-run.md) page. That page shows parameters, metrics, tags, and artifacts, including:

  - The source code notebook for that trial, saved under the **Artifacts** section. You can download this notebook and import it into the workspace (if downloading artifacts is enabled by your workspace administrator).
  - Code snippets for making predictions with the model.

To return to the experiment later, find it in the table on the [MLflow Experiments](/concepts/mlflow-experiment.md) page. ^[classification-with-automl-databricks-on-aws.md]

## Notebooks Generated by AutoML

Every AutoML experiment produces two primary types of notebooks:

1. **Data exploration notebook** – contains automated analysis of the input dataset, including distributions, correlations, and missing values.
2. **Trial notebooks** – for each hyperparameter trial, a notebook with the full training code is logged as an artifact. The notebook for the best trial is directly accessible from the results page as “View notebook for best model”.

All notebooks are stored in a `databricks_automl` folder inside the Workspace Home Folder of the user who ran the experiment. This organization makes it easy to locate and reuse the generated code across experiments. ^[classification-with-automl-databricks-on-aws.md]

## Registering and Deploying from the UI

After monitoring the experiment and selecting a candidate model, you can register it directly from the results page:

1. Click the link in the **Models** column for the desired model.
2. Click the **Register model** button to save the model to Unity Catalog or the Model Registry.
3. After registration, deploy the model to a [Model Serving](/concepts/model-serving.md) endpoint.

When serving a model built with AutoML, you may encounter a `No module named 'pandas.core.indexes.numeric'` error due to incompatible pandas versions. This can be resolved by running a provided script (`add-pandas-dependency.py`) that updates the model’s `requirements.txt` and `conda.yaml` to require `pandas==1.5.3`, then re-registering the model. ^[classification-with-automl-databricks-on-aws.md]

## Related Concepts

- AutoML – Overview of automated machine learning on Databricks
- [MLflow Experiments](/concepts/mlflow-experiment.md) – The parent container for AutoML runs
- [Unity Catalog](/concepts/unity-catalog.md) – Recommended destination for model registration
- [Model Registry](/concepts/mlflow-model-registry.md) – Alternative destination for model versioning
- Workspace Home Folder – Default storage location for AutoML outputs
- [Model Serving](/concepts/model-serving.md) – Deployment of registered models to endpoints

## Sources

- classification-with-automl-databricks-on-aws.md

# Citations

1. [classification-with-automl-databricks-on-aws.md](/references/classification-with-automl-databricks-on-aws-61813bfe.md)
