Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Enabling Observability for Llama-Index Agent in Phoenix | Arize AI Community

Phoenix Support

Enabling Observability for Llama-Index Agent in Phoenix

·Jan 20, 2025 03:24 PM

Hello, I am very new to phoenix and I trying to enable observability for an agent created with llama-index The trances of agent.chat() is working fine but the traces of agent.astream_chat() is like this. I don't know whether this is the intended behavior or it is a bug.

4 comments

· Sorted by Oldest

RunLLM
APP
·
The issue you're experiencing with agent.astream_chat() not showing traces as expected might be related to a known bug where token calculation is not performed during streaming with LlamaIndex when using certain LLMs like groqLLM. This issue was identified and resolved in a recent update to the openinference-instrumentation-llama-index package (source). To address this, ensure you are using the latest version of the openinference-instrumentation-llama-index package. If the problem persists, it might be due to the specific LLM being used, as some LLMs may not support token counting during streaming. Would you like more detailed guidance on updating or troubleshooting this issue further?Sources: - GitHub Issue 5729
💡 Hint: Mention RunLLM in the thread for followups.
Mikyo
·
Hey Mohammed, today's a day off at Arize so our responses might be a bit delayed. In general with LlamaIndex we consume the spans that they produce so my guess might be that the instrumentation on the llamaindex side has something to do with this. Astream based instrumentation is a bit more complex due to the fact that context about each "agent" needs to be propagated correctly. That being said, there might be some gaps we missed on our end. If you have any details you can provide about your setup (versions etc.) and can drop a ticket for us to take a look we'll be sure to follow up! https://github.com/Arize-ai/openinference
Mohammed E.
·
Thanks Mikyo for your reply. I am using the following versions:
arize-phoenix==5.7.0 arize-phoenix-otel==0.6.1 llama-index==0.12.7
Also, I will open a ticket at Llamaindex's side with the issue.
Mikyo
·
Thanks Mohammed! We will take a look.