In https://arize.com/docs/phoenix/tracing/integrations-tracing/litellm#setup this doc it mentions you have to run the `register` function. Is this a one-off thing with litellm, or do we have to run this function every time we want phoenix to pick up up some traces?
You only need to call the register function once per process, before instrumenting and making LLM calls. This sets up the OpenTelemetry tracer provider so Phoenix can receive traces; you do not need to call it before every LLM call鈥攋ust once at application startup or before any tracing begins. According to the Phoenix LiteLLM integration docs, after calling register, all subsequent instrumented calls will be traced automatically.Would you like more detail on how to structure this in your code?Sources:
- Phoenix LiteLLM integration docs
馃挕聽Hint: Mention RunLLM in the thread for followups.
To group multiple LLM calls, embeddings, and other spans under a single trace鈥攅ven when they occur across different files鈥攜ou should create a top-level span (e.g., a CHAIN span) at the start of the request and ensure all subsequent spans are children of this span by propagating the trace context. This is typically done by starting the top-level span and using context propagation (e.g., via context managers or explicit context passing) so all nested operations share the same trace_id. See manual instrumentation guide and OpenTelemetry context propagation for details.Would you like a code example or more detail on context propagation across files?Sources: - https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089 - https://arize-ai.slack.com/archives/C04R3GXC8HK/p1714665357.760569
馃挕聽Hint: Mention RunLLM in the thread for followups.
Hey Alex S., you might find this page helpful https://arize.com/docs/phoenix/tracing/how-to-tracing/setup-tracing/instrument-python
I tried and couldn't figure it out Xander S. https://mlops.systems/posts/2025-06-04-instrumenting-an-agentic-app-with-arize-phoenix-and-litellm.html (also cc: Mikyo via the LLM Evals course discord)
Hey Alex S. sorry the docs were confusing. Happy to lean in with John G. to help alleviate any questions you may have had. The experience you had with the lite-LLM proxy is noted. This integration was written by the litellm team to start and we鈥檝e had some difficulties maintaining it. Will alert them and see what we can do.
Hey Alex S. - thanks for pointing out the issues here, really helps us improve! I captured a few docs updates we need to make based on your feedback, and will work through those In terms of what you're looking to do, to group the spans under a top level trace, you'd need to start a span before calling your other functions. Easiest way to do this would be to add a decorator to your my_llm_application function:
import litellm
from phoenix.otel import register
tracer_provider = register(
project_name="hinbox", # Default is 'default'
auto_instrument=True, # Auto-instrument your app based on installed OI dependencies
set_global_tracer_provider=False,
batch=True,
)
tracer = tracer_provider.get_tracer(__name__)
@tracer.llm
def query_llm(prompt: str):
completion_response = litellm.completion(
model="openrouter/google/gemma-3n-e4b-it:free",
messages=[
{
"content": prompt,
"role": "user",
}
],
)
return completion_response.choices[0].message.content
@tracer.agent
def query_agent(prompt: str):
return "I am an agent."
@tracer.chain
def my_llm_application():
query1 = query_llm("What's the capital of China? Just give me the name.")
query2 = query_llm("What's the capital of Japan? Just give me the name.")
agent1 = query_agent("Who are you?")
return (query1, query2, agent1)
if __name__ == "__main__":
print(my_llm_application())That would cause any calls made within that function to be grouped under the top level span. Let me know if that doesn't work as you'd expect!
Ok yes I see that now! I'll update my notes to reflect your suggested solution. It's unclear to me why I see the same thing in both the traces and the spans tab of the dashboard, though. Shouldn't I see spans in the span tab and traces in the traces tab? I guess it's unclear to me what the difference is for Phoenix in how these two things get handled. It seems that a chain is somehow some kind of a parent object? or is it just treated that way because of how the code is constructed? I guess I'll experiment a bit more to get to the bottom of this!
And what's the equivalent of @tracer.embedding since it seems like this isn't implemented?
or @tracer.guardrails or @tracer.reranker etc
Hey Alex, Traces as the concept under "signals" is basically a unique identifier of spans (think "span" of time). See https://opentelemetry.io/docs/concepts/signals/traces/ In most cases if you filter spans by "roots" (e.g. spans that don't have parents) and or look at the collective set of "traces" they will roughly look the same. Most of the time this is the view you want when looking at telemetry. Spans are too noisy to be looking at in isolation. While the two tabs feel largely overlapping, it's a bit intentional as there's actually no real object called a trace - it's just a series of spans. You will see these abstractions in most observability platform.
"there's actually no real object called a trace - it's just a series of spans" ok that's super clarifying that that's the position you take. I think would be super helpful to say that in your docs somewhere too FWIW.
And what's the equivalent of @tracer.embedding since it seems like this isn't implemented? [9:44 AM] or @tracer.guardrails or @tracer.reranker etc
We emit spans for embedding text to vectors (like "adda"), guardrailing via thinks like guardrals or content moderation, and reranking things via things like cohere. However it's sorta rare for people to manually write these. We will have decorators for them but right now they are typically emitted from autoinstrumentors like langgraph where there are common patterns for these things.
Ok yeah I'm not using a framework so I have to manually instrument all this.