Switching from LlamaIndex to OpenAI Client: Instrumentation Questions

Swaraj R. · 2024-12-10T17:59:07.863Z

@RunLLM if i'm currently using llama index and llama index instrumentor like this at the end of my setup function LlamaIndexInstrumentor().instrument() if i wanna switch to using open ai client directly? Will i have to .instrument another instrumentor like OpenAI? Can both be instrumented at the same time?

11 comments

· Sorted by Oldest

RunLLM
APP
·
Yes, you can instrument both the LlamaIndex and OpenAI clients simultaneously. You will need to call the instrument method for each instrumentor separately. For OpenAI, you can use the OpenAIInstrumentor to instrument the OpenAI client. According to the Phoenix documentation, you can initialize the OpenAIInstrumentor before your application code to ensure that all requests are traced and reported to Phoenix.
Would you like more detail on how to set this up?
Sources: - Phoenix documentation
💡 Hint: Mention RunLLM in the thread for followups.
Swaraj R.
·
Ah interesting so if i have some parts using open ai directly, i should instrument it and uninstrument after calling?
Swaraj R.
·
Thanks Mikyo ! To clarify, you are suggesting to add the open inference OpenAIInstrumentor.instrument() to our current tracing setup? based on this
If you use our OpenAIInstrumentor you will get spans for your calls to OpenAI. I would start there. Would be happy to do follow-up enhancements to make the two work together a bit nicer.
Would it still create duplicate spans for now then?
Mikyo
·
It would right now Swaraj R. - I apologize for that. You'd get one produced by llama-index and one from the invocation of the openai client. It's a problem we need to solve. More people are stitching together frameworks and providers but llama-index workflows has made it a bit difficult since we have to rely on llama-index to track things a bit.
Mikyo
·
Keep us in the loop with your painpoints. Would love to help
Swaraj R.
·
Ah so the dupes are only for the open ai calls? not all spans right? (esp custom ones we are creating)
Mikyo
·
Yeah exactly
Swaraj R.
·
if so, then that's not so bad while we migrate
Swaraj R.
·
Thank you!
1
Mikyo
·
No problem! Yeah it will make your OpenAI calls replayable in Phoenix, which I think is a worthy tradeoff for now 😅
🙌1
Swaraj R.
·
A very big reason why i'm doing this 😅 , love the playground feature but llama index traces were showing up as simple input/output vs template, tools, etc that we have
❤️1

Switching from LlamaIndex to OpenAI Client: Instrumentation Questions | Arize AI Community

RunLLM
APP
·
Yes, you can instrument both the LlamaIndex and OpenAI clients simultaneously. You will need to call the instrument method for each instrumentor separately. For OpenAI, you can use the OpenAIInstrumentor to instrument the OpenAI client. According to the Phoenix documentation, you can initialize the OpenAIInstrumentor before your application code to ensure that all requests are traced and reported to Phoenix.
Would you like more detail on how to set this up?
Sources: - Phoenix documentation
💡 Hint: Mention RunLLM in the thread for followups.
Swaraj R.
·
Ah interesting so if i have some parts using open ai directly, i should instrument it and uninstrument after calling?
Swaraj R.
·
Thanks Mikyo ! To clarify, you are suggesting to add the open inference OpenAIInstrumentor.instrument() to our current tracing setup? based on this
If you use our OpenAIInstrumentor you will get spans for your calls to OpenAI. I would start there. Would be happy to do follow-up enhancements to make the two work together a bit nicer.
Would it still create duplicate spans for now then?
Mikyo
·
It would right now Swaraj R. - I apologize for that. You'd get one produced by llama-index and one from the invocation of the openai client. It's a problem we need to solve. More people are stitching together frameworks and providers but llama-index workflows has made it a bit difficult since we have to rely on llama-index to track things a bit.
Mikyo
·
Keep us in the loop with your painpoints. Would love to help
Swaraj R.
·
Ah so the dupes are only for the open ai calls? not all spans right? (esp custom ones we are creating)
Mikyo
·
Yeah exactly
Swaraj R.
·
if so, then that's not so bad while we migrate
Swaraj R.
·
Thank you!
1
Mikyo
·
No problem! Yeah it will make your OpenAI calls replayable in Phoenix, which I think is a worthy tradeoff for now 😅
🙌1
Swaraj R.
·
A very big reason why i'm doing this 😅 , love the playground feature but llama index traces were showing up as simple input/output vs template, tools, etc that we have
❤️1