---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: dd09ff47e5b5571492d8618b2b71d5be5b941221ad85c91da2c08a48b46e7eef
  pageDirectory: concepts
  sources:
    - arize-phoenix-scorers-databricks-on-aws.md
  confidence: 0.9
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - evaluation-dataset-structure-for-genai
    - EDSFG
  citations:
    - file: arize-phoenix-scorers-databricks-on-aws.md
title: Evaluation Dataset Structure for GenAI
description: A structured dataset format with inputs, outputs, and expectations (context) fields used for MLflow GenAI evaluation
tags:
  - data-format
  - evaluation
  - mlflow
timestamp: "2026-06-19T17:35:49.672Z"
---

---  
title: Evaluation Dataset Structure for GenAI  
summary: The expected schema for GenAI evaluation datasets including inputs, outputs, and expectations  
sources:  
  - arize-phoenix-scorers-databricks-on-aws.md  
kind: concept  
createdAt: "2026-06-18T14:27:39.560Z"  
updatedAt: "2026-07-17T10:00:00.000Z"  
tags:  
  - data-format  
  - evaluation  
  - LLM  
aliases:  
  - evaluation-dataset-structure-for-genai  
  - EDSFG  
confidence: 0.85  
provenanceState: merged  
inferredParagraphs: 1  
---

# Evaluation Dataset Structure for GenAI

An **evaluation dataset** for [GenAI](/concepts/mlflow-genai-evaluate-api.md) applications is a structured collection of examples used to assess the quality of an agent or LLM’s responses. The dataset is provided as a list of dictionaries, where each element contains the inputs the application receives, the outputs it generates, and optional expectations against which judges or scorers measure performance. ^[arize-phoenix-scorers-databricks-on-aws.md]

## Core Fields

Each evaluation example includes one or more of the following fields:

### `inputs`

The **`inputs`** field contains the data passed to the application under test. For question-answering tasks, `inputs` is typically a dictionary with a `"query"` string. ^[arize-phoenix-scorers-databricks-on-aws.md]

```python
{"inputs": {"query": "What is MLflow?"}}
```

### `outputs`

The **`outputs`** field holds the response generated by the application. When the dataset is produced by `mlflow.genai.evaluate()` with a prediction function, MLflow automatically populates this field. For offline evaluation, you can pre‑populate `outputs` with ground‑truth responses. ^[arize-phoenix-scorers-databricks-on-aws.md]

```python
{"outputs": "MLflow is an open-source AI engineering platform for agents and LLMs."}
```

### `expectations`

The **`expectations`** field provides reference information that judges or scorers can use to evaluate quality. It commonly contains a `"context"` key with supporting facts or a document relevant to the input. Expectations are optional; they serve as a ground truth for metrics like faithfulness or relevance. ^[arize-phoenix-scorers-databricks-on-aws.md]

```python
{"expectations": {"context": "MLflow is an ML platform for experiment tracking and model deployment."}}
```

## Dataset Format

The dataset is a list of dictionaries, passed directly to the `data` parameter of `mlflow.genai.evaluate()`. Each dictionary can contain any subset of the core fields. ^[arize-phoenix-scorers-databricks-on-aws.md]

```python
eval_dataset = [
    {
        "inputs": {"query": "What is MLflow?"},
        "outputs": "MLflow is an open-source AI engineering platform for agents and LLMs.",
        "expectations": {
            "context": "MLflow is an ML platform for experiment tracking and model deployment."
        },
    },
    {
        "inputs": {"query": "How do I track experiments?"},
        "outputs": "You can use mlflow.start_run() to begin tracking experiments.",
        "expectations": {
            "context": "MLflow provides APIs like mlflow.start_run() for experiment tracking."
        },
    },
]
```

## Usage with Scorers

The evaluation dataset is used with scorers such as `Hallucination` and `Relevance` from the Arize Phoenix integration. These scorers read the `inputs`, `outputs`, and `expectations` fields to compute quality metrics. ^[arize-phoenix-scorers-databricks-on-aws.md]

```python
results = mlflow.genai.evaluate(
    data=eval_dataset,
    scorers=[
        Hallucination(model="databricks:/databricks-gpt-5-mini"),
        Relevance(model="databricks:/databricks-gpt-5-mini"),
    ],
)
```

## Best Practices

- **Structure inputs consistently** so that your scorers expect the same keys as the dataset.
- **Include expectations** when you want automated judges to check factual accuracy or context adherence. ^[arize-phoenix-scorers-databricks-on-aws.md]

## Related Concepts

- [MLflow Evaluation](/concepts/mlflow-evaluation-ui.md) — The API that consumes evaluation datasets
- [Custom Judges](/concepts/custom-judges.md) — LLM-based scorers that use dataset fields
- Arize Phoenix — Third‑party scorer integration for hallucination and relevance

## Sources

- arize-phoenix-scorers-databricks-on-aws.md

# Citations

1. [arize-phoenix-scorers-databricks-on-aws.md](/references/arize-phoenix-scorers-databricks-on-aws-53f4b817.md)
