Switching Between Multi-Project Configurations in Litellm Framework

Saeed M. · 2025-04-29T16:53:23.142Z

Hi, I'm trying to figure out how to switch projects on the fly. Basically, I've registered multiple projects when I initiate my app. Now, I want to send different llm calls to different projects for better separation , how can I do that? BTW, I'm using litellm framework.

RunLLM
APP
·
To switch projects dynamically in Phoenix, especially when using the litellm framework, you can utilize the using_project context manager. This allows you to temporarily associate spans with different projects within the scope of your function or block of code. Here's how you can achieve this: 1. Use the using_project Context Manager: This context manager allows you to switch the project for all spans created within its context. This is particularly useful in environments like notebooks but can be adapted for other use cases.2. Instrument litellm: Ensure that your litellm calls are instrumented to send traces to Phoenix.Here's an example of how you can set this up: LBKSimport os from phoenix.trace import using_project from openinference.instrumentation.litellm import LiteLLMInstrumentor import litellmLBKS# Set up LiteLLM instrumentation LiteLLMInstrumentor().instrument()LBKS# Example usage of using_project context manager def make_llm_call(project_name, model, messages): with using_project(project_name): response = litellm.completion(model=model, messages=messages) print(f"Response for project {project_name}: {response}")LBKS# Example LLM calls to different projects make_llm_call("project1", "gpt-3.5-turbo", [{"content": "Hello, how are you?", "role": "user"}]) make_llm_call("project2", "gpt-3.5-turbo", [{"content": "What's the weather today?", "role": "user"}])LBKSThis code will send the LLM calls to different projects as specified by the project_name parameter in the using_project context manager.Would you like more details on any specific part?Sources: - GitHub Issue #3207
💡 Hint: Mention RunLLM in the thread for followups.
RunLLM
APP
·
If you tell me the right answer to this question, I'll make sure I don't make the same mistake twice!
Feedback
Type the right answer here...
↵ Please 'enter' to submit
Xander S.
·
Saeed M. You can use openinference.instrumentation.using_project. Alternatively, you can create multiple tracers, one for each project.
Xander S.
·
Let us know if this page helps https://docs.arize.com/phoenix/tracing/how-to-tracing/setup-tracing/setup-projects
Saeed M.
·
It says it's possible to use using_project context manager in notebooks. However, I'm trying to do it in my production environment.
Xander S.
·
Saeed M. In that case, I would recommend creating separate tracer providers with distinct project resources.
Saeed M.
·
I would, but I want to assign each function to a speicific project and I don't find a way to do that. Additionally, for sessions, all your examples are for openai, it's not clear how to create sessions with litellm framework.
Xander S.
·
I would, but I want to assign each function to a speicific project and I don't find a way to do that.
How many projects do you intend to use?
Saeed M.
·
Let's say 4 to 5, each for different features
Xander S.
·
It sounds like you want traces generated by a specific function to go to a particular project.
Xander S.
·
And you are using LiteLLM auto-instrumentation within your individual functions.
Saeed M.
·
correct
Xander S.
·
Okay, is your goal primarily to split out your traces into logical spaces for distinct analysis? Maybe you can help me understand the need for separate projects in your case. Tags are another option for grouping traces within a single project, for example.
Saeed M.
·
That is correct. I'm looking for logical spaces for distinct analysis as you described.
Saeed M.
·
I can use tags if splitting projects is not yet available.

23 comments