Using Gemini, Claude, and Other LLMs with Phoenix for RAG Eval

Sangram · 2024-10-01T09:22:12.473Z

Hey 🔒[private user] 🔒[private user] we are using Phoenix as a main eval framework for RAG eval. Now we want to use the Gemini, Claude, and other open-source LLMs for the evaluation. so how can we use those LLMs? By default phoenix uses GPT? We tried as per the documentation but we are unable to find it helpful?? So if you have any notebook or code of how to use other LLMs for eval apart from GPT, please share and help us with this?

Using Gemini, Claude, and Other LLMs with Phoenix for RAG Eval | Arize AI Community

3 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix supports various LLMs for evaluation, including Gemini and Claude, through the LiteLLM library. By default, Phoenix uses GPT models, but you can configure it to use other models.To use Gemini, Claude, and other LLMs, you can follow these steps: 1. Install LiteLLM: Ensure you have the LiteLLM package installed. 2. Set Environment Variables: Configure the necessary API keys and environment variables for the LLMs you want to use. 3. Use the completion Method: Call the completion method from LiteLLM to interact with the desired LLM.Here is an example code snippet for using different LLMs: LBKSfrom litellm import completionLBKS# Set environment variables for the LLMs os.environ["OPENAI_API_KEY"] = "your_openai_key" os.environ["COHERE_API_KEY"] = "your_cohere_key" os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_key"LBKSmessages = [{"content": "Hello, how are you?", "role": "user"}]LBKS# OpenAI call response_openai = completion(model="gpt-3.5-turbo", messages=messages)LBKS# Cohere call response_cohere = completion(model="command-nightly", messages=messages)LBKS# Anthropic call response_anthropic = completion(model="claude-instant-1", messages=messages)LBKSprint(response_openai, response_cohere, response_anthropic)LBKSFor more detailed instructions, you can refer to the LiteLLM documentation and the Phoenix API documentation.Would you like more detailed steps or a specific example for a particular LLM?Sources: - https://github.com/Arize-ai/phoenix/issues/2140 - https://github.com/Arize-ai/phoenix/issues/2117
💡 Hint: Mention RunLLM in the thread for followups.
Sangram
·
How can i use open source models? like llama 3.2, Gemma using GROQ api???
Dustin N.
·
Hi Sangram! Depending on the interface you’re using to call the model, you can either use LiteLLM as a wrapper, or possibly write your own model wrapper to pass into Phoenix. If the latter is something you need to do (for instance, you host your model behind a custom API) let me know and I’m happy to help

RunLLM
APP
·
Phoenix supports various LLMs for evaluation, including Gemini and Claude, through the LiteLLM library. By default, Phoenix uses GPT models, but you can configure it to use other models.To use Gemini, Claude, and other LLMs, you can follow these steps: 1. Install LiteLLM: Ensure you have the LiteLLM package installed. 2. Set Environment Variables: Configure the necessary API keys and environment variables for the LLMs you want to use. 3. Use the completion Method: Call the completion method from LiteLLM to interact with the desired LLM.Here is an example code snippet for using different LLMs: LBKSfrom litellm import completionLBKS# Set environment variables for the LLMs os.environ["OPENAI_API_KEY"] = "your_openai_key" os.environ["COHERE_API_KEY"] = "your_cohere_key" os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_key"LBKSmessages = [{"content": "Hello, how are you?", "role": "user"}]LBKS# OpenAI call response_openai = completion(model="gpt-3.5-turbo", messages=messages)LBKS# Cohere call response_cohere = completion(model="command-nightly", messages=messages)LBKS# Anthropic call response_anthropic = completion(model="claude-instant-1", messages=messages)LBKSprint(response_openai, response_cohere, response_anthropic)LBKSFor more detailed instructions, you can refer to the LiteLLM documentation and the Phoenix API documentation.Would you like more detailed steps or a specific example for a particular LLM?Sources: - https://github.com/Arize-ai/phoenix/issues/2140 - https://github.com/Arize-ai/phoenix/issues/2117
💡 Hint: Mention RunLLM in the thread for followups.
Sangram
·
How can i use open source models? like llama 3.2, Gemma using GROQ api???
Dustin N.
·
Hi Sangram! Depending on the interface you’re using to call the model, you can either use LiteLLM as a wrapper, or possibly write your own model wrapper to pass into Phoenix. If the latter is something you need to do (for instance, you host your model behind a custom API) let me know and I’m happy to help