---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 730290d0930bae69781a8a7a937aaf6407f0d599dd4801cdcf8b0e4884c9e211
  pageDirectory: concepts
  sources:
    - arize-phoenix-scorers-databricks-on-aws.md
  confidence: 0.9
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - llm-evaluation-dataset-schema
    - LEDS
  citations:
    - file: arize-phoenix-scorers-databricks-on-aws.md
title: LLM Evaluation Dataset Schema
description: The structured dataset format used in mlflow.genai.evaluate, requiring inputs, outputs, and expectations (with context) fields
tags:
  - mlflow
  - data-format
  - llm-evaluation
timestamp: "2026-06-18T10:48:35.708Z"
---

# LLM Evaluation Dataset Schema

The **LLM evaluation dataset schema** defines the structure of evaluation datasets used with `mlflow.genai.evaluate()` to assess LLM-generated outputs using third-party scorers such as those provided by Arize Phoenix. The schema is a list of dictionaries, where each dictionary represents a single evaluation record containing the model's input, the model's output, and optional expectations against which the output is scored.^[arize-phoenix-scorers-databricks-on-aws.md]

## Schema Fields

Each record in the evaluation dataset is a dictionary with the following keys:

- **`inputs`** (dict, required): A dictionary that describes the prompt or query sent to the LLM. In the standard form, this dictionary contains a single key `"query"` whose value is the text of the user's question.^[arize-phoenix-scorers-databricks-on-aws.md]
- **`outputs`** (string, required): The response generated by the LLM for the given query. This field holds the model's output text.^[arize-phoenix-scorers-databricks-on-aws.md]
- **`expectations`** (dict, optional): A dictionary of reference information used by scorers to evaluate the quality of the output. For example, a `"context"` key can provide the ground-truth context that a hallucination or relevance scorer compares against the output.^[arize-phoenix-scorers-databricks-on-aws.md]

## Example

The following example, taken from the Arize Phoenix scorers documentation, shows a two-record evaluation dataset that matches the schema described above:

```python
eval_dataset = [
    {
        "inputs": {"query": "What is MLflow?"},
        "outputs": "MLflow is an open-source AI engineering platform for agents and LLMs.",
        "expectations": {
            "context": "MLflow is an ML platform for experiment tracking and model deployment."
        },
    },
    {
        "inputs": {"query": "How do I track experiments?"},
        "outputs": "You can use mlflow.start_run() to begin tracking experiments.",
        "expectations": {
            "context": "MLflow provides APIs like mlflow.start_run() for experiment tracking."
        },
    },
]
```

This dataset is passed to `mlflow.genai.evaluate()` along with a list of [[scorers]] (e.g., `Hallucination` and `Relevance`) to produce evaluation metrics for each record.^[arize-phoenix-scorers-databricks-on-aws.md]

## Usage Notes

- The schema is designed to work with any scorer that expects `inputs` and `outputs`; the `expectations` dictionary may vary by scorer (some scorers may require different keys or omit this field entirely).
- The structure shown is the canonical form used by MLflow's GenAI evaluation API; extensions such as including additional metadata keys are possible but may not be recognized by all scorers.

## Related Concepts

- [MLflow GenAI](/concepts/mlflow-3-for-genai.md)
- [LLM Evaluation](/concepts/llm-as-a-judge-evaluation.md)
- [[Scorers]]
- [Hallucination Scorer](/concepts/hallucination-scorer.md)
- [Relevance Scorer](/concepts/relevance-scorer.md)
- [mlflow.genai.evaluate()](/concepts/mlflowgenaievaluate.md)

## Sources

- arize-phoenix-scorers-databricks-on-aws.md

# Citations

1. [arize-phoenix-scorers-databricks-on-aws.md](/references/arize-phoenix-scorers-databricks-on-aws-53f4b817.md)
