---
type: concept
x-llmwiki:
  schemaVersion: "0.1"
  contentHash: 6223a6a78075e36ef7bba3f2294bdd2e3115ba8641f1a3abd7e3c003513f6e25
  pageDirectory: concepts
  sources:
    - automatic-tracing-databricks-on-aws.md
  confidence: 1
  provenanceState: extracted
  freshnessStatus: unverified
  aliases:
    - multi-llm-call-tracing-pattern
    - MCTP
    - sequential-multi-llm-call-tracing-pattern
    - SMCTP
  citations:
    - file: automatic-tracing-databricks-on-aws.md
title: Multi-LLM Call Tracing Pattern
description: Design pattern where automatic tracing captures multiple sequential LLM calls within a single workflow, illustrated by an analysis-then-response pattern with conditional model routing.
tags:
  - mlflow
  - tracing
  - llm
  - workflow-pattern
timestamp: "2026-06-18T10:51:07.399Z"
---

# Multi-LLM Call Tracing Pattern

**Multi-LLM call tracing** is a pattern for capturing and visualizing sequences of multiple large language model (LLM) invocations within a single application workflow. It enables developers to observe how different LLM calls interact, how results from one call influence subsequent calls, and to debug or monitor complex multi-step AI pipelines. ^[automatic-tracing-databricks-on-aws.md]

## Overview

Modern AI applications often require more than a single LLM call. Common scenarios include:

- **Sequential decision-making**: A first call analyzes a query, and a second call generates a response based on that analysis.
- **Multi-agent systems**: Different agents with distinct roles or models collaborate on a task.
- **Hybrid workflows**: Combine direct LLM API calls with framework abstractions (e.g., LangChain chains) and custom business logic.

The Multi-LLM Call Tracing Pattern captures all these invocations in a unified trace tree, making it easy to understand end-to-end behavior and identify performance bottlenecks or logical errors. ^[automatic-tracing-databricks-on-aws.md]

## Enabling Multi-LLM Call Tracing

MLflow provides two complementary mechanisms for tracing multiple LLM calls:

### Automatic Tracing

With [Automatic Tracing](/concepts/automatic-tracing.md), you enable tracing for a supported library (such as OpenAI or LangChain) by calling `mlflow.<library>.autolog()`. Every call made through that library's SDK is automatically captured as a span in the trace. ^[automatic-tracing-databricks-on-aws.md]

To trace multiple LLM frameworks in the same workflow, call `autolog()` for each one:

```python
import mlflow

# Enable auto-tracing for both OpenAI and LangChain
mlflow.openai.autolog()
mlflow.langchain.autolog()
```

^[automatic-tracing-databricks-on-aws.md]

### Manual Tracing

Use the `@mlflow.trace` decorator to create custom spans around business logic that sits between LLM calls. This captures any processing or state management that happens outside the traced library calls. ^[automatic-tracing-databricks-on-aws.md]

```python
@mlflow.trace(span_type=SpanType.CHAIN)
def process_user_query(query: str):
    # First LLM call: analyze the query
    analysis = client.chat.completions.create(...)
    # Second LLM call: generate response based on analysis
    response = client.chat.completions.create(...)
    return response
```

^[automatic-tracing-databricks-on-aws.md]

### Combining Both Approaches

You can combine manual and automatic tracing in the same workflow. The `@mlflow.trace` decorator adds a custom parent span, while the `autolog()` calls automatically generate child spans for each LLM invocation within that parent. This creates a single unified trace showing the full flow. ^[automatic-tracing-databricks-on-aws.md]

## Trace Structure

A multi-LLM trace produces a hierarchical span tree:

- **Parent span**: The top-level workflow function, annotated with `@mlflow.trace`.
- **Child spans**: One span per LLM invocation, automatically generated by the framework's `autolog()` integration.

For example, a workflow that makes two sequential OpenAI calls produces a trace with one parent span (`process_user_query`) and two child spans (one for each `chat.completions.create` call). ^[automatic-tracing-databricks-on-aws.md]

## Prerequisites

To use the Multi-LLM Call Tracing Pattern, you need:

- **MLflow 3** (recommended) or MLflow 2.x with the GenAI tracing plugin. ^[automatic-tracing-databricks-on-aws.md]
- The appropriate integration packages installed (e.g., `openai>=1.0.0`, `langchain`). ^[automatic-tracing-databricks-on-aws.md]
- Credentials configured for each LLM provider you intend to use. ^[automatic-tracing-databricks-on-aws.md]
- A valid [MLflow tracking URI](/concepts/mlflow-tracking-uri.md) set to your Databricks workspace. ^[automatic-tracing-databricks-on-aws.md]

## Example: Sequential Decision-Making

The following pattern is useful when your application needs to make sequential decisions, such as analyzing a query first and then generating a response based on that analysis: ^[automatic-tracing-databricks-on-aws.md]

```python
import mlflow
from openai import OpenAI
from mlflow.entities import SpanType

mlflow.openai.autolog()

client = OpenAI()

@mlflow.trace(span_type=SpanType.CHAIN)
def process_user_query(query: str):
    # First LLM call: Analyze the query
    analysis = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[...]
    )
    analysis_result = analysis.choices[0].message.content

    # Second LLM call: Generate response based on analysis
    if "factual" in analysis_result.lower():
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "system", "content": "Provide a factual response."}, ...]
        )
    else:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "system", "content": "Provide a creative response."}, ...]
        )
    return response.choices[0].message.content

result = process_user_query("Tell me about the history of AI")
```

## Example: Multi-Provider Hybrid Workflow

Combine OpenAI direct calls, LangChain chains, and custom logic in a single trace: ^[automatic-tracing-databricks-on-aws.md]

```python
import mlflow
from openai import OpenAI
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

mlflow.openai.autolog()
mlflow.langchain.autolog()

client = OpenAI()

@mlflow.trace(span_type=SpanType.CHAIN)
def multi_provider_workflow(query: str):
    # First, use OpenAI directly for initial analysis
    analysis = client.chat.completions.create(...)
    topics = analysis.choices[0].message.content

    # Then use LangChain for structured response
    llm = ChatOpenAI(model="gpt-4o-mini")
    prompt = ChatPromptTemplate.from_template(
        "Based on these topics: {topics}\nGenerate a response to: {query}"
    )
    chain = prompt | llm
    response = chain.invoke({"topics": topics, "query": query})
    return response
```

## Benefits

- **Unified observability**: One trace shows the complete workflow across multiple LLM calls.
- **Per-call visibility**: Inspect inputs, outputs, latency, and token usage for each LLM invocation.
- **Debugging context**: Understand how the output of one call affects the next.
- **Framework flexibility**: Works with any combination of supported tracing integrations. ^[automatic-tracing-databricks-on-aws.md]

## Related Concepts

- [Automatic Tracing](/concepts/automatic-tracing.md) — Automatically capture LLM calls without manual instrumentation
- [Manual Tracing](/concepts/manual-tracing.md) — Add custom spans around business logic with `@mlflow.trace`
- [MLflow Tracing](/concepts/mlflow-tracing.md) — The underlying tracing infrastructure for GenAI applications
- Span — The fundamental unit of a trace, representing a single operation
- Trace Tree — The hierarchical structure of parent and child spans
- [MLflow Tracking](/concepts/mlflow-tracking.md) — The experiment tracking system that stores traces

## Sources

- automatic-tracing-databricks-on-aws.md

# Citations

1. [automatic-tracing-databricks-on-aws.md](/references/automatic-tracing-databricks-on-aws-ff944f8c.md)
