Issue with Importing Functions from litellm in Phoenix Setup

Saeed M. · 2025-04-30T20:51:00.184Z

I ran into this issue and I want your help. I've experimented with phoenix and found out that it doesn't send traces when we directly import acompletion and aembedding functions from litellm module; however, when I import litellm and use litellm.aecompletion and litellm.aembedding it works. However, using the latter approach would break our tests. So, I'm not sure what is the right approach. I'm not quite sure what is the diff between direct import acompletion and importing module and calling one of its functions. Is it initialization? Is there monkey patching somewhere in the process?

Issue with Importing Functions from litellm in Phoenix Setup | Arize AI Community

8 comments

· Sorted by Oldest

RunLLM
APP
·
The issue you're experiencing is likely due to how the LiteLLMInstrumentor in the openinference-instrumentation-litellm package instruments the litellm functions. According to the LiteLLM Instrumentation documentation, the instrumentation works by monkey-patching the litellm module functions. This means that the functions are wrapped with additional logic to capture traces when they are called.When you import functions directly (e.g., from litellm import acompletion), you bypass the monkey-patching because the instrumentation modifies the functions on the litellm module itself, not the individual function references. Therefore, using litellm.aecompletion and litellm.aembedding ensures that the instrumented versions of the functions are called, capturing the necessary traces.Would you like more detail on how to address this in your tests or any other aspect? Sources: - GitHub Issue on LiteLLM Instrumentation - LiteLLM Instrumentation documentation
💡 Hint: Mention RunLLM in the thread for followups.
Saeed M.
·
That makes sense. Is there a way to just import those specific methods and then force LiteLLMInstrumentor to do its magic on them without importing the entire package?
Saeed M.
·
RunLLM
RunLLM
APP
·
To ensure that the LiteLLMInstrumentor can instrument specific methods like acompletion and aembedding without importing the entire litellm package, you would need to ensure that the instrumentation is applied after the functions are imported. However, the current implementation of LiteLLMInstrumentor relies on monkey-patching the entire litellm module, which means it modifies the functions directly on the module.Unfortunately, based on the available data sources, there is no direct support for instrumenting only specific functions without importing the entire module. The instrumentation is designed to work by modifying the functions on the litellm module itself, so importing the module is necessary for the instrumentation to take effect.If you want to avoid importing the entire module in your tests, you might consider creating a wrapper or a mock that imports the module, applies the instrumentation, and then exposes only the functions you need. This way, you can control the scope of the import while still benefiting from the instrumentation.Would you like more detail on how to implement such a wrapper or mock?Sources: - https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/src/openinference/instrumentation/litellm/init.py - https://github.com/Arize-ai/phoenix/issues/4963
💡 Hint: Mention RunLLM in the thread for followups.
Saeed M.
·
Thanks John. I used instrumentor before importing those methods and looks like it's working. Although I get this warning in my logs which I don't know if it's going to cause any problem down the road: Attempting to instrument while already instrumented.
# Instrument first LiteLLMInstrumentor().instrument( completion_methods=["acompletion"], embedding_methods=["aembedding"] ) # Then import the instrumented methods from litellm import acompletion, aembedding
Saeed M.
·
🔒[private user] BTW, do you have any paid support for self hosting implementation by any chance?
Saeed M.
·
The former one (technical support for self-host Phoenix on our infra). I'm trying to set up phoenix in our k8s.
Saeed M.
·
I'm available after 2:30pm ET on Monday. That works for me.

RunLLM
APP
·
The issue you're experiencing is likely due to how the LiteLLMInstrumentor in the openinference-instrumentation-litellm package instruments the litellm functions. According to the LiteLLM Instrumentation documentation, the instrumentation works by monkey-patching the litellm module functions. This means that the functions are wrapped with additional logic to capture traces when they are called.When you import functions directly (e.g., from litellm import acompletion), you bypass the monkey-patching because the instrumentation modifies the functions on the litellm module itself, not the individual function references. Therefore, using litellm.aecompletion and litellm.aembedding ensures that the instrumented versions of the functions are called, capturing the necessary traces.Would you like more detail on how to address this in your tests or any other aspect? Sources: - GitHub Issue on LiteLLM Instrumentation - LiteLLM Instrumentation documentation
💡 Hint: Mention RunLLM in the thread for followups.
Saeed M.
·
That makes sense. Is there a way to just import those specific methods and then force LiteLLMInstrumentor to do its magic on them without importing the entire package?
Saeed M.
·
RunLLM
RunLLM
APP
·
To ensure that the LiteLLMInstrumentor can instrument specific methods like acompletion and aembedding without importing the entire litellm package, you would need to ensure that the instrumentation is applied after the functions are imported. However, the current implementation of LiteLLMInstrumentor relies on monkey-patching the entire litellm module, which means it modifies the functions directly on the module.Unfortunately, based on the available data sources, there is no direct support for instrumenting only specific functions without importing the entire module. The instrumentation is designed to work by modifying the functions on the litellm module itself, so importing the module is necessary for the instrumentation to take effect.If you want to avoid importing the entire module in your tests, you might consider creating a wrapper or a mock that imports the module, applies the instrumentation, and then exposes only the functions you need. This way, you can control the scope of the import while still benefiting from the instrumentation.Would you like more detail on how to implement such a wrapper or mock?Sources: - https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/src/openinference/instrumentation/litellm/init.py - https://github.com/Arize-ai/phoenix/issues/4963
💡 Hint: Mention RunLLM in the thread for followups.
Saeed M.
·
Thanks John. I used instrumentor before importing those methods and looks like it's working. Although I get this warning in my logs which I don't know if it's going to cause any problem down the road: Attempting to instrument while already instrumented.
# Instrument first LiteLLMInstrumentor().instrument( completion_methods=["acompletion"], embedding_methods=["aembedding"] ) # Then import the instrumented methods from litellm import acompletion, aembedding
Saeed M.
·
🔒[private user] BTW, do you have any paid support for self hosting implementation by any chance?
Saeed M.
·
The former one (technical support for self-host Phoenix on our infra). I'm trying to set up phoenix in our k8s.
Saeed M.
·
I'm available after 2:30pm ET on Monday. That works for me.