---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 2da19baa13a488b70a72f172527a0600c96895e809af00b29fd4c06c0ab8a8b7
  pageDirectory: concepts
  sources:
    - human-feedback-in-mlflow-databricks-on-aws.md
  confidence: 0.98
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - feedback-vs-expectation-in-mlflow
    - FVEIM
  citations:
    - file: human-feedback-in-mlflow-databricks-on-aws.md
title: Feedback vs Expectation in MLflow
description: "Two distinct assessment types: Feedback evaluates the app's actual outputs (ratings, comments on what was generated), while Expectation defines the desired or correct outcome (ground truth) that the app should have produced, useful for creating evaluation datasets."
tags:
  - mlflow
  - human-feedback
  - data-model
  - evaluation
timestamp: "2026-06-19T19:07:00.871Z"
---

# Feedback vs Expectation in MLflow

**Feedback** and **Expectation** are the two types of assessments that MLflow uses to capture human input on GenAI application traces. Both are stored as [Assessments](/concepts/assessments.md) attached to individual [[MLflow Trace|MLflow Traces]], but they serve fundamentally different purposes in the evaluation lifecycle. ^[human-feedback-in-mlflow-databricks-on-aws.md]

## Overview

MLflow captures human feedback as assessments attached to your GenAI app's traces, allowing you to collect ratings, comments, and ground-truth expectations from developers, end users, and domain experts. Human feedback complements automated evaluation and is used to create datasets for [LLM Judges](/concepts/llm-judges.md) and scorers, keeping them aligned with human expert judgment. ^[human-feedback-in-mlflow-databricks-on-aws.md]

## Feedback

**Feedback** evaluates your app's *actual outputs* or intermediate steps. It answers questions like, "Was the agent's response good?" Feedback assesses what the app produced, such as ratings or comments, and provides qualitative insights into the app's performance. ^[human-feedback-in-mlflow-databricks-on-aws.md]

Feedback is inherently tied to the specific output generated by the app for a given input. Different runs or different app versions may produce different outputs, and feedback captures the quality of whatever was actually generated. ^[human-feedback-in-mlflow-databricks-on-aws.md]

## Expectation

**Expectation** defines the *desired or correct outcome* (ground truth) that your app *should have produced*. For example, this could be "the ideal response" to a user's query. For a given input, the Expectation is always the same, regardless of what the app actually generated. ^[human-feedback-in-mlflow-databricks-on-aws.md]

Expectations define what the app should generate and are useful for creating [Evaluation Datasets](/concepts/evaluation-datasets.md). They serve as a reference standard against which actual outputs can be compared. ^[human-feedback-in-mlflow-databricks-on-aws.md]

## Key Differences

| Aspect | Feedback | Expectation |
|--------|----------|-------------|
| **What it captures** | Evaluation of actual outputs | Desired or correct outcome |
| **Question it answers** | "Was the response good?" | "What should the response have been?" |
| **Relationship to input** | Varies with each run/output | Always the same for a given input |
| **Primary use** | Qualitative insights, ratings | Ground truth, evaluation datasets |

## Data Model

Both Feedback and Expectation are stored as assessments attached to individual MLflow Traces. Assessments can be attached to the entire Trace or to a specific span within the Trace. This links feedback directly to a specific user query and your GenAI app's outputs and logic. ^[human-feedback-in-mlflow-databricks-on-aws.md]

## Collection Sources

MLflow helps you collect both feedback and expectations from three main sources, each tailored for a different use case in your GenAI app's lifecycle: ^[human-feedback-in-mlflow-databricks-on-aws.md]

- **Developer feedback**: During development, you can directly annotate traces to track quality notes and mark specific examples for future reference or regression testing.
- **Domain expert feedback**: Subject matter experts provide structured feedback on app outputs and define expectations for correct responses. Their evaluations help define what high-quality responses look like and are invaluable for aligning LLM judges with nuanced business requirements.
- **End-user feedback**: In production, capture feedback from users interacting with your live application to gain insights into real-world performance.

## Related Concepts

- [Assessments](/concepts/assessments.md) — The data model that stores both feedback and expectations
- [[MLflow Trace|MLflow Traces]] — The trace data to which assessments are attached
- [Automated Evaluation](/concepts/automated-evaluation-and-monitoring.md) — Complements human feedback in the evaluation workflow
- [Evaluation Datasets](/concepts/evaluation-datasets.md) — Created from expectations for systematic evaluation
- [LLM Judges](/concepts/llm-judges.md) — Aligned with human expert judgment using feedback data
- [Review App](/concepts/mlflow-review-app.md) — Interface for collecting domain expert feedback and expectations

## Sources

- human-feedback-in-mlflow-databricks-on-aws.md

# Citations

1. [human-feedback-in-mlflow-databricks-on-aws.md](/references/human-feedback-in-mlflow-databricks-on-aws-5bbf5fdf.md)
