---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: cae10262035edfa6117f4a4d420a795dac60b7beef08b0ce60008fabe5523613
  pageDirectory: concepts
  sources:
    - arize-phoenix-scorers-databricks-on-aws.md
  confidence: 0.85
  provenanceState: merged
  freshnessStatus: unverified
  aliases:
    - expectations-based-llm-evaluation-dataset
    - ELED
  citations:
    - file: arize-phoenix-scorers-databricks-on-aws.md
title: Expectations-based LLM Evaluation Dataset
description: An evaluation dataset structure for MLflow GenAI that includes 'inputs', 'outputs', and 'expectations' (with 'context') to provide ground-truth reference for scorers.
tags:
  - llm-evaluation
  - dataset
  - mlflow
timestamp: "2026-06-19T14:04:05.454Z"
---

# Expectations-based LLM Evaluation Dataset

**Expectations-based LLM Evaluation Dataset** refers to a dataset format used with [MLflow GenAI Evaluation](/concepts/mlflow-genai-evaluation.md) where each record includes an `inputs` field, an `outputs` field (the model's response), and an `expectations` field that contains reference or context information used by [LLM-as-a-Judge](/concepts/llm-as-a-judge.md) scorers to evaluate response quality.

## Overview

In expectations-based evaluation, the dataset provides structured test cases that include both the model's generated output and the expected context or reference material. Scorers — such as [Hallucination Scorer](/concepts/hallucination-scorer.md) and [Relevance Scorer](/concepts/relevance-scorer.md) from Arize Phoenix — use the `expectations` field to assess whether the model's output is factually grounded and relevant to the provided context. ^[arize-phoenix-scorers-databricks-on-aws.md]

## Dataset Structure

Each record in an expectations-based evaluation dataset contains three key fields:

- **`inputs`**: The query or prompt presented to the LLM.
- **`outputs`**: The response generated by the LLM being evaluated.
- **`expectations`**: Reference information, such as ground-truth context, that judges use to score the output.

The `outputs` field distinguishes this format from prediction-only evaluation, where the model generates the response during evaluation. Here, outputs are pre-computed and provided in the dataset. ^[arize-phoenix-scorers-databricks-on-aws.md]

### Example

```python
eval_dataset = [
    {
        "inputs": {"query": "What is MLflow?"},
        "outputs": "MLflow is an open-source AI engineering platform for agents and LLMs.",
        "expectations": {
            "context": "MLflow is an ML platform for experiment tracking and model deployment."
        },
    },
    {
        "inputs": {"query": "How do I track experiments?"},
        "outputs": "You can use mlflow.start_run() to begin tracking experiments.",
        "expectations": {
            "context": "MLflow provides APIs like mlflow.start_run() for experiment tracking."
        },
    },
]
```

^[arize-phoenix-scorers-databricks-on-aws.md]

## Common Scorers Using Expectations

### Hallucination Scorer

The `Hallucination` scorer checks whether the model's output contains information that is not supported by the provided `expectations` context. It flags statements that contradict or go beyond the reference material. ^[arize-phoenix-scorers-databricks-on-aws.md]

### Relevance Scorer

The `Relevance` scorer assesses how well the output relates to the input query and the provided context in `expectations`. It measures whether the response addresses the user's question appropriately. ^[arize-phoenix-scorers-databricks-on-aws.md]

## Usage

To evaluate using an expectations-based dataset with Phoenix scorers:

```python
import mlflow
from mlflow.genai.scorers.phoenix import Hallucination, Relevance

results = mlflow.genai.evaluate(
    data=eval_dataset,
    scorers=[
        Hallucination(model="databricks:/databricks-gpt-5-mini"),
        Relevance(model="databricks:/databricks-gpt-5-mini"),
    ],
)
```

^[arize-phoenix-scorers-databricks-on-aws.md]

## Comparison with Prediction-Based Evaluation

In prediction-based LLM evaluation, the dataset contains only `inputs` (and optionally `expectations`), and the model generates `outputs` during evaluation. In contrast, expectations-based evaluation pre-supplies the `outputs`, making it suitable for cases where responses are generated externally or need to be scored against fixed reference material without re-running the model. ^[arize-phoenix-scorers-databricks-on-aws.md]

## Related Concepts

- [LLM-as-a-Judge Evaluation](/concepts/llm-as-a-judge-evaluation.md) — Using LLMs to score outputs against criteria
- Hallucination detection — Detecting unsupported content in LLM outputs
- Response relevance evaluation — Measuring how well output addresses the query
- [Evaluation dataset formats](/concepts/evaluation-dataset-for-prompts.md) — Different structures for MLflow evaluation data
- [Arize Phoenix integration](/concepts/arize-phoenix-scorers-integration.md) — Third-party scorers available in MLflow
- Context-based evaluation — Scoring grounded in reference material

## Sources

- arize-phoenix-scorers-databricks-on-aws.md

# Citations

1. [arize-phoenix-scorers-databricks-on-aws.md](/references/arize-phoenix-scorers-databricks-on-aws-53f4b817.md)
