You can view the number of tokens used (prompt, completion, and total) for each LLM invocation directly in the Phoenix UI. The token count is displayed in the trace details, with a tooltip that breaks down prompt and completion tokens separately, provided the model or integration supplies this information in the trace data. See <https://github.com/Arize-ai/phoenix/blob/main/app/src/components/trace/TokenCount.tsx|TokenCount.tsx> and <https://github.com/Arize-ai/phoenix/blob/main/tutorials/integrations/tracing_and_evals_with_mongodb_and_llama_index.ipynb|UI documentation>.
If token counts are missing, it may be due to the model/provider not supplying them (especially in streaming mode), or the integration not passing them through. For best results, ensure your LLM integration provides token usage in the expected format; Phoenix will display it automatically if present. For more, see <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1720508176.999739|Slack discussion>.
Would you like more detail on troubleshooting missing token counts or on how to ensure your integration provides them?
Sources:
- <https://github.com/Arize-ai/phoenix/blob/main/app/src/components/trace/TokenCount.tsx|TokenCount.tsx>
- <https://github.com/Arize-ai/phoenix/blob/main/tutorials/integrations/tracing_and_evals_with_mongodb_and_llama_index.ipynb|Tracing and Evals with MongoDB and Llama Index>
- <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1720508176.999739|Slack discussion>
