---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 6c8e9a0280820de9a7c24544e98fa7932921a2c4d70d2af6465a72a18edad79c
  pageDirectory: concepts
  sources:
    - genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md
  confidence: 1
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - agent-labeling-and-review
    - Review and Agent Labeling
    - ALAR
  citations:
    - file: genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md
title: Agent Labeling and Review
description: Workflows for viewing labeling sessions, assigning reviewers to traces, and inspecting labeling schemas for feedback criteria such as ratings and comments
tags:
  - labeling
  - review
  - genai
  - feedback
timestamp: "2026-06-19T18:59:01.171Z"
---

# Agent Labeling and Review

**Agent Labeling and Review** refers to the process of viewing, inspecting, and providing feedback on traces generated by AI agents within MLflow experiments. It is a core part of agent observability, enabling teams to assess agent performance through structured labeling sessions, human feedback, and review workflows. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

## Overview

In the context of [MLflow](/concepts/mlflow.md) and [GenAI](/concepts/mlflow-genai-evaluate-api.md) application development, labeling and review allows users to examine agent behavior, assign ratings and comments to traces, and track who is responsible for reviewing specific interactions. This process is essential for understanding agent quality, identifying failure patterns, and improving agent performance over time. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

## Key Concepts

- **Labeling Sessions**: Structured periods during which traces are reviewed and assigned feedback. Labeling sessions are created and managed within MLflow experiments. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]
- **Labeling Schemas**: Define the feedback criteria used during review, such as rating scales, comment fields, and expectations for agent behavior. These schemas guide reviewers in providing consistent, actionable feedback. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]
- **Review Assignments**: Each labeling session can track which reviewers are assigned to evaluate specific traces, enabling accountability and workflow management. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]
- **Human Feedback**: Ratings, comments, and other annotations provided by human reviewers during the labeling process. This feedback can be used alongside [LLM judge](/concepts/llm-judges.md) scores and programmatic checks to form a complete picture of agent quality. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

## Using Genie Code for Labeling and Review

[Genie Code](/concepts/genie-code.md) provides a natural language interface for exploring labeling and review data within an MLflow experiment. Through conversational queries, users can: ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

- View existing labeling sessions and identify who is assigned to review specific traces.
- Inspect labeling schemas to understand the feedback criteria in use.
- Discover traces with low user feedback scores and investigate what went wrong in those conversations.
- Correlate labeling results with trace data, evaluation scores, and other metrics.

### Example Questions

Users can ask Genie Code questions such as: ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

- "Which sessions have the lowest user feedback scores, and what went wrong in those conversations?"
- "View labeling sessions and who's assigned to review traces"
- "Inspect labeling schemas to understand feedback criteria such as ratings, comments, and expectations"

## Relationship to Evaluation

Labeling and review is one component of a broader [Agent Evaluation](/concepts/mlflow-agent-evaluation.md) workflow. While [LLM Judges](/concepts/llm-judges.md) and programmatic scorers provide automated quality assessments, human labeling captures nuanced feedback that automated systems may miss. Combining these approaches gives a more complete view of agent performance. ^[genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md]

Reviewers can use labeling data alongside evaluation scores, trace analysis, and metrics to identify patterns and prioritize improvements.

## Related Concepts

- [Genie Code](/concepts/genie-code.md) — Natural language interface for exploring MLflow experiment data
- [MLflow Tracing](/concepts/mlflow-tracing.md) — End-to-end observability for GenAI agents
- [Agent Evaluation](/concepts/mlflow-agent-evaluation.md) — Automated and manual assessment of agent quality
- [LLM Judge](/concepts/llm-judges.md) — Automated scoring using large language models
- [Human Feedback](/concepts/mlflow-human-feedback-collection.md) — Ratings and comments provided by human reviewers
- [Labeling Schema](/concepts/labeling-schema.md) — Defines the criteria and structure for feedback collection
- Trace Analysis — Inspecting individual agent execution flows
- [Prompt Registry](/concepts/prompt-registry.md) — Managing prompts in Unity Catalog

## Sources

- genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md

# Citations

1. [genie-code-for-agent-observability-and-evaluation-databricks-on-aws.md](/references/genie-code-for-agent-observability-and-evaluation-databricks-on-aws-deaf4f1f.md)
