---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 35378c88538529712a3c7a7e09c69dae3dba5279188bd5d6dacd8b55660c88de
  pageDirectory: concepts
  sources:
    - collect-user-feedback-databricks-on-aws.md
  confidence: 0.95
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - end-user-feedback-collection-patterns
    - EUFCP
  citations:
    - file: collect-user-feedback-databricks-on-aws.md
title: End User Feedback Collection Patterns
description: "Two approaches for linking user feedback to GenAI traces: using MLflow's auto-generated trace IDs or custom client request IDs."
tags:
  - mlflow
  - genai
  - feedback
  - pattern
timestamp: "2026-06-19T09:16:53.741Z"
---

# End User Feedback Collection Patterns

**End User Feedback Collection Patterns** refer to the structured methods for capturing real-world user feedback on GenAI application outputs within MLflow. By logging feedback as assessments on [[MLflow Trace|MLflow Traces]], teams gain ground truth signals to monitor quality, guide improvements, and build evaluation datasets from production data.

## Why Collect User Feedback?

User feedback provides essential real-world quality signals that automated metrics alone cannot capture. It enables teams to:

- Understand how actual users perceive the application’s outputs.
- Identify patterns in negative feedback to guide iterative development.
- Build high-quality evaluation datasets from production interactions (see [Build Evaluation Datasets](/concepts/evaluation-datasets.md)).
- Track satisfaction metrics over time and across user segments.
- Leverage feedback data for Model Fine-Tuning. ^[collect-user-feedback-databricks-on-aws.md]

## Feedback Data Model

In MLflow, user feedback is represented by the **Feedback** entity, a type of [Assessment](/concepts/assessments.md) that can be attached to a trace or a specific span. Each feedback entry stores:

- **Value**: The actual feedback – boolean (e.g., thumbs up/down), numeric rating, text, or structured data.
- **Source**: An `AssessmentSource` object identifying the provider (e.g., `HUMAN` with a user ID, or `LLM_JUDGE` for automated evaluation).
- **Rationale**: Optional explanation for the feedback.
- **Metadata**: Additional context such as timestamps or custom attributes. ^[collect-user-feedback-databricks-on-aws.md]

This data model integrates seamlessly with MLflow’s evaluation and monitoring capabilities. For the full schema, see the [Feedback Entity](/concepts/feedback-object.md) documentation. ^[collect-user-feedback-databricks-on-aws.md]

## Two Collection Patterns

In production, you need to link user feedback to a specific trace. Two patterns achieve this:

1. **Using MLflow trace IDs** – Rely on the trace ID automatically generated by MLflow. This is the simplest approach.
2. **Using client request IDs** – Generate your own unique ID for each request and reference it later when logging feedback.

Both patterns follow the same lifecycle: generate or retrieve an identifier during the initial request, expose it to the client after the response is delivered, and use the `mlflow.log_feedback()` API to attach the feedback to the original trace. ^[collect-user-feedback-databricks-on-aws.md]

### Pattern 1: MLflow Trace ID

The backend retrieves the current trace ID (e.g., via `mlflow.get_current_active_span().trace_id`) and includes it in the API response to the client. The client then sends the trace ID back when the user submits feedback. The source provides a complete implementation using FastAPI and React. ^[collect-user-feedback-databricks-on-aws.md]

### Pattern 2: Client Request ID

Instead of exposing the internal trace ID, the application generates its own request ID (e.g., a UUID) at the start of processing. This ID is returned to the client and later used to look up the corresponding trace when feedback arrives. The source mentions this as an alternative but does not provide a detailed implementation. ^[collect-user-feedback-databricks-on-aws.md]

## Handling Streaming Responses

When responses are streamed (e.g., via Server-Sent Events), the trace ID is not available until the stream completes. This changes the feedback collection pattern:

1. The backend starts an MLflow trace and streams tokens to the client.
2. Once the stream finishes, the backend sends a final event containing the trace ID.
3. The frontend enables feedback controls only after receiving the trace ID.

Key considerations for streaming include:

- Use a consistent event format with a `type` field (e.g., `token`, `done`, `error`).
- Buffer management to handle partial SSE messages in the frontend.
- Error handling and trace logging for failures.
- Small delays between tokens (`asyncio.sleep(0.01)`) to prevent overwhelming clients. ^[collect-user-feedback-databricks-on-aws.md]

The source provides a full FastAPI streaming endpoint and a React hook that manages streaming content and feedback submission.

## Analyzing Feedback Data

Once feedback is collected, you can query traces and analyze patterns using the `MlflowClient`.

### Getting Traces with Feedback

Retrieve traces from a time window using `client.search_traces()`. Then fetch full trace details (including assessments) with `client.get_trace()`. Filter for assessments whose name matches your feedback key (e.g., `"user_feedback"`). ^[collect-user-feedback-databricks-on-aws.md]

### Analyzing Patterns

Compute metrics such as feedback rate (percentage of traces with feedback) and positive rate. For multi-dimensional feedback (e.g., separate ratings for accuracy, helpfulness, relevance), aggregate scores per dimension. The source shows Python functions that calculate these metrics. ^[collect-user-feedback-databricks-on-aws.md]

## Best Practices

- **Design your UI to handle trace ID timing**: In streaming, disable feedback buttons until the trace ID is received.
- **Use consistent event structures**: Distinguish between tokens, completion, and errors.
- **Reset state per interaction**: Clear previous trace ID and feedback status at the start of each new request.
- **Log errors to traces**: Capture failures for debugging.
- **Consider performance**: Batch tokens if they arrive too quickly and implement backpressure for slow clients. ^[collect-user-feedback-databricks-on-aws.md]

## Related Concepts

- [MLflow Tracing](/concepts/mlflow-tracing.md)
- [AssessmentSource](/concepts/assessmentsource-entity.md)
- [Build Evaluation Datasets](/concepts/evaluation-datasets.md)
- [Production Monitoring for GenAI](/concepts/production-quality-monitoring-mlflow-genai.md)
- [MLflow Evaluation](/concepts/mlflow-evaluation-ui.md)
- Streaming Responses

## Sources

- collect-user-feedback-databricks-on-aws.md

# Citations

1. [collect-user-feedback-databricks-on-aws.md](/references/collect-user-feedback-databricks-on-aws-0b0ba83c.md)
