Skip to main content

Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Phoenix Support

Separate Input and Output Token Count for LLM Cost Calculation

·Jul 18, 2024 05:18 AM

Hi, Can we get the input tokens and output tokens count seperately to calculate the llm cost ?

5 comments

· Sorted by Oldest

Separate Input and Output Token Count for LLM Cost Calculation | Arize AI Community

Roger Y.
·
you can use llm.token_count.prompt for input and llm.token_count.completion for output
Ruchika B.
·
Ok thanks
Ruchika B.
·
and second thing while this isntruemntation I am getting the session_id as well in the response which is not reuired for us due to security issue so how can we avoid getting that.
Roger Y.
·
Are you using LangChain? Where does session_id appear?
Roger Y.
·
I don’t think Phoenix collects this info. It’s probably collected by LangChain.