Troubleshooting Token Usage in Phoenix Tracing Setup

If you are not seeing token usage or cost data in Phoenix traces with both OpenAIInstrumentor and AutogenAgentChatInstrumentor enabled, the most likely cause is that the OpenAI LLM calls are not being properly instrumented, or the token count attributes are not being attached to the correct spans. Phoenix cost tracking requires that each LLM span includes llm.token_count.prompt, llm.token_count.completion, llm.token_count.total, llm.model_name, and llm.provider attributes, which are automatically captured only if the OpenAI instrumentation is correctly applied to the actual OpenAI client calls in your application. If these attributes are missing, cost aggregation will not work as expected. This is confirmed in the cost tracking documentation and OpenInference OpenAI Instrumentation README.A common issue is that the OpenAI client may be imported or instantiated before the OpenAIInstrumentor().instrument() call, which prevents the instrumentor from monkey-patching the client and capturing token usage. Ensure that the OpenAI instrumentor is initialized before any OpenAI client is created or used. Also, verify that your OpenAI SDK version is at least 1.26 and that, if you are streaming completions, you set stream_options={"include_usage": True} to get token counts (FAQ). If you are using AutoGen, make sure it is actually calling the OpenAI Python SDK and not another backend, as only supported providers will be auto-instrumented for token tracking. If you still do not see token counts, try running a minimal script with only OpenAI instrumentation to confirm that token usage is captured, then incrementally add back AutoGen and other components.Would you like a step-by-step checklist or a minimal working example to help debug this further?Sources: - Phoenix Cost Tracking Documentation - OpenInference OpenAI Instrumentation README - Phoenix Tracing FAQ

Having models_usage (with prompt_tokens and completion_tokens) inside your messages array is not sufficient for Phoenix cost tracking—Phoenix requires these token counts to be present as top-level span attributes: llm.token_count.prompt, llm.token_count.completion, and llm.token_count.total (source). If the token usage is only nested within message payloads and not promoted to span attributes, Phoenix will not aggregate or display cost data.To resolve this, ensure your instrumentation extracts models_usage from each message and sets the corresponding span attributes. This may require a custom callback or patch to your tracing logic to map models_usage fields to the required OpenInference attributes on the relevant LLM spans (source).Would you like a code example for extracting and setting these attributes?Sources: - Phoenix Cost Tracking Documentation - Phoenix GitHub Issue #8488

Certainly! Here’s a step-by-step solution to ensure Phoenix cost tracking works by extracting token usage from your message payloads and setting them as span attributes.---Step 1: Understand the RequirementPhoenix cost tracking requires that each LLM span has the following attributes set at the span level (not just inside message payloads): - llm.token_count.prompt - llm.token_count.completion - llm.token_count.total - llm.model_name - llm.providerPhoenix will not aggregate or display cost data if these are missing, even if token counts are present in nested fields like models_usage (source, GitHub Issue #8488).---Step 2: Extract Token Usage from Message PayloadsSuppose you have a message structure like this: LBKSmessage = { "models_usage": { "prompt_tokens": 275, "completion_tokens": 68 }, # ... other fields ... }LBKSYou need to extract these values and compute the total: LBKSprompt_tokens = message["models_usage"]["prompt_tokens"] completion_tokens = message["models_usage"]["completion_tokens"] total_tokens = prompt_tokens + completion_tokensLBKS---Step 3: Set Span Attributes ManuallyIf you are using OpenTelemetry directly or customizing spans, set these attributes on the current span: LBKSfrom opentelemetry import traceLBKSspan = trace.get_current_span() span.set_attribute("llm.token_count.prompt", prompt_tokens) span.set_attribute("llm.token_count.completion", completion_tokens) span.set_attribute("llm.token_count.total", total_tokens) span.set_attribute("llm.model_name", model_name) # Set this to your model's name span.set_attribute("llm.provider", provider_name) # Set this to your provider, e.g., "openai"LBKS---Step 4: Integrate with Your Tracing LogicIf you’re using a callback or custom handler (e.g., in LangChain, AutoGen, or your own framework), add this logic where you process LLM responses. For example, in a callback’s on_llm_end method: LBKSdef on_llm_end(self, response, **kwargs): # Extract token usage from response/messages models_usage = response.get("models_usage") if models_usage: prompt_tokens = models_usage.get("prompt_tokens", 0) completion_tokens = models_usage.get("completion_tokens", 0) total_tokens = prompt_tokens + completion_tokensLBKS span = trace.get_current_span() span.set_attribute("llm.token_count.prompt", prompt_tokens) span.set_attribute("llm.token_count.completion", completion_tokens) span.set_attribute("llm.token_count.total", total_tokens) span.set_attribute("llm.model_name", "gpt-3.5-turbo") # Example span.set_attribute("llm.provider", "openai") # ExampleLBKS

If you have multiple messages, repeat this for each relevant LLM-generated message.---Step 5: Verify in PhoenixAfter running your application, check the Phoenix UI. The LLM spans should now display token usage and cost data. If not, inspect the raw span attributes to ensure the fields are present at the top level.---Step 6: Automate for All LLM Calls (Optional)References and Examples- Phoenix Cost Tracking Documentation - Phoenix GitHub Issue #8488 - Manual Instrumentation Example---Example Code BlockLBKSfrom opentelemetry import traceLBKSdef set_llm_token_attributes(message, model_name, provider_name): models_usage = message.get("models_usage") if models_usage: prompt_tokens = models_usage.get("prompt_tokens", 0) completion_tokens = models_usage.get("completion_tokens", 0) total_tokens = prompt_tokens + completion_tokensLBKS span = trace.get_current_span() span.set_attribute("llm.token_count.prompt", prompt_tokens) span.set_attribute("llm.token_count.completion", completion_tokens) span.set_attribute("llm.token_count.total", total_tokens) span.set_attribute("llm.model_name", model_name) span.set_attribute("llm.provider", provider_name)LBKSCall this function after each LLM call, passing the message and model/provider info.---Let me know if you need a more specific example for your framework or want help integrating this into your codebase!Sources: - Phoenix Cost Tracking Documentation - Phoenix GitHub Issue #8488 - Manual Instrumentation Example

When you use a custom FilteringSpanProcessor to filter out spans like 'autogen create', 'autogen send', etc., you are likely removing the LLM spans where token counts are attached. Phoenix cost tracking relies on these LLM spans to aggregate and display token usage; if they are filtered out, token data will not appear in the dashboard. This is expected behavior, as documented in the Phoenix span filtering guide: filtering out spans removes their data from export and downstream analytics.If you want token counts to be visible, avoid filtering out LLM-related spans or ensure that the spans you keep include the required token count attributes.Would you like guidance on how to filter only non-LLM spans while preserving token tracking?Sources: - Phoenix span filtering guide

You should remove any filters that exclude LLM spans, as these are the spans where token usage and cost attributes are attached. In your FilteringSpanProcessor, the current filter excludes spans whose names start with: - 'autogen create' - 'autogen send' - 'autogen process' - 'autogen publish' - 'autogen ack' - 'create_agent' - 'autogen consume' - 'autogen receive'and spans with exact names: - '_process_message' - '_handle_response' - '_format_message'However, if any of your LLM (OpenAI) calls or their token-counting logic are instrumented under these names, filtering them out will remove the token usage from Phoenix. To ensure token tracking works, do NOT filter out spans that represent LLM calls (often named like 'openai.chat.completions', 'openai.completions', or similar). You can safely filter out only spans that are confirmed to be internal, non-LLM, and do not contain token usage attributes.Recommended change: - Remove any prefix or exact name from unwanted_prefixes and unwanted_exact_names if it matches the span name of your LLM calls or their direct parents. - If unsure, temporarily disable the filter and inspect span names in Phoenix to identify which ones contain token usage, then adjust your filter to keep those.For more, see the Phoenix span filtering guide.Would you like help identifying the exact span names to keep for OpenAI LLM calls?Sources: - Phoenix span filtering guide

If you now see token counts but cost remains zero, it means Phoenix is receiving token usage but cannot match your span’s model name and provider to a pricing entry. For cost to be calculated, the span must have the following attributes: llm.token_count.prompt, llm.token_count.completion, llm.token_count.total, llm.model_name, and llm.provider, and the model/provider combination must match an entry in Phoenix’s model pricing table (source).Check your span attributes in Phoenix and confirm: - llm.model_name matches a known model (e.g., gpt-4.1 or gpt-4). - llm.provider is set (e.g., openai). - If using a custom or new model, add or update its pricing in Phoenix under Settings → Models, ensuring the name pattern and provider match the span attributes (source).Would you like step-by-step instructions for adding a custom model price entry?Sources: - Phoenix Cost Tracking Documentation - Phoenix GitHub Issue #8465