---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 073b7940ab05ad58b6422c00e73816ccd35791210b8b7be932a74c824e99f255
  pageDirectory: concepts
  sources:
    - custom-judges-databricks-on-aws.md
  confidence: 0.98
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - template-variables-for-judge-prompts
    - TVFJP
  citations:
    - file: custom-judges-databricks-on-aws.md
title: Template Variables for Judge Prompts
description: The four allowed template variables in make_judge prompts — inputs, outputs, expectations, and trace — that provide access to agent data during evaluation.
tags:
  - prompt-engineering
  - mlflow
  - genai
timestamp: "2026-06-19T18:03:12.286Z"
---

# Template Variables for Judge Prompts

**Template variables** are placeholders embedded in the natural-language instructions of a [custom judge](/concepts/custom-judges.md) to reference agent data at evaluation time. They allow the judge to access the agent's inputs, outputs, expected behaviors, or complete execution trace. Custom judges are created using the [`make_judge()`](https://mlflow.org/docs/3.4.0/api_reference/python_api/mlflow.genai.html#mlflow.genai.judges.make_judge) function from MLflow. ^[custom-judges-databricks-on-aws.md]

## Allowed Template Variables

Four template variables are available for use in judge instructions. Your instructions must include at least one, but you are not required to use all four. ^[custom-judges-databricks-on-aws.md]

| Variable | Description |
|----------|-------------|
| `{{ inputs }}` | The input data provided to the agent |
| `{{ outputs }}` | The output data generated by your agent |
| `{{ expectations }}` | Ground-truth answers or expected outcomes |
| `{{ trace }}` | The complete execution trace of your agent, including tool calls and intermediate steps |

## Usage Rules

- Your judge instructions **must** contain at least one of the four supported template variables. A prompt without any template variable is invalid. ^[custom-judges-databricks-on-aws.md]
- Custom variable names such as `{{ question }}`, `{{ context }}`, or any name outside the allowed set are **not permitted**. Using them causes a validation error, because the framework enforces a fixed set of variables to ensure consistent behavior and prevent template injection attacks. ^[custom-judges-databricks-on-aws.md]

## Trace-Based Judge Considerations

When you include `{{ trace }}` in the instructions, the judge becomes a *trace-based judge*. Trace-based judges analyze the full execution trace — including tool invocations, reasoning steps, and performance data — by autonomously exploring the trace with Model Context Protocol (MCP) tools. This capability enables validation of multi-step workflows and tool-usage patterns. ^[custom-judges-databricks-on-aws.md]

For trace-based judges to analyze the complete trace, the `model` argument must be specified in `make_judge()`. Recommended models include `databricks:/databricks-gpt-5-mini`, `databricks:/databricks-gpt-5`, `databricks:/databricks-gpt-oss-120b`, and `databricks:/databricks-claude-opus-4-5`. The model can be served by Foundation Model APIs, External Model Serving Endpoints, or Custom Model Serving Endpoints. ^[custom-judges-databricks-on-aws.md]

## Best Practices for Using Template Variables

- **Be explicit about the output format.** Judge instructions that use template variables should clearly specify whether the judge returns a boolean, a categorical value, or a numeric score. For example, list specific categorical values such as `'fully_resolved'`, `'partially_resolved'`, `'needs_follow_up'`, or explicitly state the judge should return `true` or `false`. For numeric scores, specify the scoring range and what each score means. ^[custom-judges-databricks-on-aws.md]
- **Structure complex evaluations.** For tasks that examine multiple aspects (e.g., relevance, correctness, and tool usage), break the instructions into clear sections such as *What to evaluate*, *What information to examine*, *How to make the judgment*, and *What format to return*. ^[custom-judges-databricks-on-aws.md]

## Related Concepts

- [Custom Judges](/concepts/custom-judges.md) – How to create and use LLM-based evaluators
- make_judge()|Make Judge API – The `make_judge()` function that accepts template variables
- [Trace-Based Evaluation](/concepts/mlflow-trace-based-evaluation.md) – Evaluation using execution traces and the `{{ trace }}` variable
- [Built-in LLM Judges](/concepts/built-in-llm-judges.md) – Pre-defined judges that serve as alternatives to custom judges
- Align Judges with Human Feedback – Improving judge accuracy with expert annotations

## Sources

- custom-judges-databricks-on-aws.md

# Citations

1. [custom-judges-databricks-on-aws.md](/references/custom-judges-databricks-on-aws-7a56fe4f.md)
