---
title: Human feedback in MLflow | Databricks on AWS
source: https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/
ingestedAt: "2026-06-18T08:15:55.861Z"
---

MLflow captures human feedback as assessments attached to your GenAI app's traces, allowing you to collect ratings, comments, and ground-truth expectations from developers, end users, and domain experts. Human feedback complements [automated evaluation](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/concepts/eval-harness). Use it to create datasets for LLM judges and scorers, and to keep them aligned with human expert judgment.

## Data model overview[​](#data-model-overview "Direct link to Data model overview")

MLflow stores human feedback as [_Assessments_](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/dev-annotations), attached to individual [MLflow Traces](https://docs.databricks.com/aws/en/mlflow3/genai/tracing/). This links feedback directly to a specific user query and your GenAI app's outputs and logic.

There are 2 assessment types:

1.  [_Feedback_](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/dev-annotations): Evaluates your app's _actual outputs_ or intermediate steps. For example, it answers questions like, "Was the agent's response good?". Feedback assesses what the app produced, such as ratings or comments. Feedback assesses what was generated by the app and provides qualitative insights.
2.  [_Expectation_](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/dev-annotations): Defines the _desired or correct outcome_ (ground truth) that your app _should have produced_. For example, this could be "The ideal response" to a user's query. For a given input, the Expectation is always the same. Expectations define what the app should generate and are useful for creating [evaluation datasets](https://docs.databricks.com/aws/en/mlflow3/genai/eval-monitor/concepts/eval-datasets),

Assessments can be attached to the entire Trace or a specific span within the Trace.

For more detail about tracing, see [Trace concepts](https://docs.databricks.com/aws/en/mlflow3/genai/tracing/tracing-101).

## How to collect feedback[​](#how-to-collect-feedback "Direct link to How to collect feedback")

MLflow helps you collect feedback from three main sources. Each source is tailored for a different use case in your GenAI app's lifecycle. While feedback comes from different personas, the underlying data model is the same for all personas.

### Developer feedback[​](#developer-feedback "Direct link to Developer feedback")

During development, you can directly annotate traces. This is useful to track quality notes as you build and mark specific examples for future reference or regression testing.

To learn how to annotate feedback during development, see [Label during development](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/dev-annotations).

### Domain expert feedback[​](#domain-expert-feedback "Direct link to domain-expert-feedback")

Engage subject matter experts to provide structured feedback on your app's outputs and define expectations for correct responses. Their detailed evaluations help define what high-quality responses look like for your specific use case and are invaluable for aligning LLM judges with nuanced business requirements.

![Review app preview hero image.](https://assets.docs.databricks.com/_static/images/generative-ai/review-app/review-app-hero.gif)

MLflow provides two approaches for collecting domain expert feedback using the Review App:

**Interactive testing with Chat UI**: Experts interact with your deployed app in real-time through a chat interface, providing immediate feedback on responses as they test conversational flows. This approach is ideal for "vibe checks" and qualitative validation before production deployment. To learn more, see [Test an app version with the Chat UI](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/expert-feedback/live-app-testing).

**Labeling existing traces**: Experts systematically review and label traces that have already been captured from your app. This approach is ideal for structured evaluation sessions where experts assess specific examples and define ground truth expectations. To learn more, see [Label existing traces](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/expert-feedback/label-existing-traces).

### End-user feedback[​](#end-user-feedback "Direct link to End-user feedback")

In production, capture feedback from users interacting with your live application. This provides crucial insights into real-world performance, helping you identify problematic queries that need fixing and highlight successful interactions to preserve during future updates. MLflow provides tools to capture, store, and analyze feedback directly from the users of your deployed applications.

To learn how to collect end user feedback, refer to the [collect end user feedback](https://docs.databricks.com/aws/en/mlflow3/genai/tracing/collect-user-feedback/) guide in the tracing section.

## Next steps[​](#next-steps "Direct link to Next steps")

*   [Get started with collecting human feedback](https://docs.databricks.com/aws/en/mlflow3/genai/getting-started/human-feedback) - Step through this holistic tutorial demonstrating common ways to collect human feedback.
*   [Label during development](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/dev-annotations) - Start annotating traces to track quality during development.
*   [Test an app version with the Chat UI](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/expert-feedback/live-app-testing) - Test your app interactively using a live chat interface.
*   [Label existing traces](https://docs.databricks.com/aws/en/mlflow3/genai/human-feedback/expert-feedback/label-existing-traces) - Set up systematic expert review processes.
