Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Tracking Input and Output Tokens for Cost Analysis in Queries | Arize AI Community

Phoenix Support

Tracking Input and Output Tokens for Cost Analysis in Queries

·Jan 02, 2025 07:58 PM

Is there a way to get the total input / output tokens in the traces table? Given that input and output tokens are priced differently, it would be helpful to be able to quantify the price per query going through the system

9 comments

· Sorted by Oldest

RunLLM
APP
·
Yes, you can obtain the total input and output tokens in the traces table. According to this GitHub pull request, Phoenix has added a feature to calculate the total token count for trace pages, which includes both input (prompt) and output (completion) tokens. This allows you to quantify the cost per query by summing these token counts.Would you like more detail on how to implement this or access these token counts?Sources: - this GitHub pull request
💡 Hint: Mention RunLLM in the thread for followups.
Ben M.
·
Looks like the api supports getting these values separated but it’s just not surfaced in the UI https://github.com/Arize-ai/phoenix/blob/main/app/schema.graphql#L1278-L1280
Mikyo
·
Hey Ben M. good point - you are right that this is available on the API as well as via a tooltip. Is it that you want an explicit column to view and sort by these or?
Ben M.
·
I can look at it at a per trace level, but I would love for the 2 to be broken out in the header. Right now, it just shows total tokens for the time period and I expected a similar tool tip
Ben M.
·
This is the spot I was hoping for the additional data
✅1
Mikyo
·
Got it - makes total sense Ben M. - I'm tracking it here and we should have a fix out soon https://github.com/Arize-ai/phoenix/issues/5872
❤️1
Mikyo
·
A small fix should be in tonight's build https://arize-ai.slack.com/archives/C04RSEQ27QQ/p1735863028952579
Ben M.
·
As always, Mikyo, you're great! Thank you so much
1
Ben M.
·
Upgraded this morning. It’s great!
❤️1