Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Troubleshooting Token Format Issues with AWS Bedrock and Arize AI | Arize AI Community

Phoenix Support

Troubleshooting Token Format Issues with AWS Bedrock and Arize AI

·Nov 12, 2024 05:05 AM

i just connect aws bedrock with arize ai phoenix but the token are not been shown i don't understand why i think it due to the format the token are in but i don't know how would i change the format to get the token can you help me with this thnx

13 comments

· Sorted by Oldest

RunLLM
APP
·
To address the issue of tokens not being displayed when connecting AWS Bedrock with Arize AI Phoenix, ensure that the token count data is correctly formatted and passed to Phoenix. According to the test_instrumentor.py, the token counts should be included in the span attributes using LLM_TOKEN_COUNT_PROMPT, LLM_TOKEN_COUNT_COMPLETION, and LLM_TOKEN_COUNT_TOTAL. Ensure these attributes are correctly set in your implementation.If the token counts are not appearing, verify that the response from AWS Bedrock includes the token count headers (x-amzn-bedrock-input-token-count and x-amzn-bedrock-output-token-count) and that these values are being extracted and passed to Phoenix. If you need more detailed guidance, let me know. Sources: - test_instrumentor.py
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
hey Mohit V., which bedrock model are you using?
Xander S.
·
can you provide a minimal working example to reproduce?
Mohit V.
·
i am using anthropic.claude-3-5-sonnet-20240620-v1:0 but not with python with javascript
Xander S.
·
got it
Mohit V.
·
i am using langchain.js
Xander S.
·
thanks Mohit V.. i am unsure exactly what coverage of token counts we have for the js langchain instrumentor. i've filed an issue here.
Mohit V.
·
thank you very much
Mohit V.
·
can we have a meet so we can fix this issue if you are free
Xander S.
·
hey Mohit V., the team will triage and prioritize the issue in github 🙂 the thing that would accelerate the process would be a code snippet to reproduce.
Parker S.
·
Hey Mohit V., if you're able to share a snippet that would be nice, if not, just curious if you are using streaming or not?
Mohit V.
·
we have made the custom class for langchaininstrumentation and fixed the issue thanks for the help
Parker S.
·
Hey Mohit V., we've just released a patch to fix this in our langchain instrumentor if you're interested in using it feel free to bump to 1.0.1!
🔥1