Does Phoenix Support LLM-as-Judges in GCP Model Garden?

Naveena C. · 2025-06-04T19:03:28.961Z

Hello, does Phoenix support choosing LLM-as-judges from models in GCP Model Garden or models deployed on endpoints in a GCP project?

Does Phoenix Support LLM-as-Judges in GCP Model Garden? | Arize AI Community

11 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix does not natively support selecting LLM-as-judge models directly from GCP Model Garden or arbitrary GCP endpoints. However, Phoenix supports Google's Gemini models via the Vertex AI Python SDK, allowing you to use Gemini LLMs for evaluation by authenticating with gcloud and using the GeminiModel class in Phoenix Evals. There is an open feature request to support the new google-genai SDK, but it is not yet implemented. Direct support for arbitrary GCP Model Garden models or custom endpoints is not mentioned in the data sources—only Vertex AI Gemini is supported at this time (feat: Add support for Google's Gemini models via Vertex python sdk, feature request: support google-genai as an eval model).Would you like more detail on how to use Gemini models with Phoenix, or on possible workarounds for custom endpoints?Sources: - feat: Add support for Google's Gemini models via Vertex python sdk - feature request: support google-genai as an eval model
💡 Hint: Mention RunLLM in the thread for followups.
Naveena C.
·
Yes, possible workarounds for custom endpoints would be helpful since it will help meet privacy and security related compliances.
Naveena C.
·
This will help in running generations on the fly too for certain models.
Xander S.
·
Hey Naveena C., we don't natively support Vertex AI in our playground, although please feel free to file us an enhancement request!
Naveena C.
·
Hi Xander S.! I am not looking for playground code but want to setup batch evaluation jobs using LLM evaluators that are hosted within my GCP project.
Xander S.
·
Got it! You should be able to use our LiteLLM model for that.
Xander S.
·
Actually, it looks like we have a native Vertex model https://arize.com/docs/phoenix/evaluation/how-to-evals/evaluation-models#vertexai
Xander S.
·
Can you give that a go to see if it achieves your aim?
Naveena C.
·
I checked LiteLLM earlier but that will send the hits to the actual LLM provider, right? I want to use self-hosted models as evaluators.

Xander S.

Not sure if this answers the question, but LiteLLM will provide a wrapper around the VertexAI SDK. I expect it should be possible to use VertexAIModel for your purpose.

        project (str, optional): The default project to use when making API calls. Defaults to None.
        location (str, optional): The default location to use when making API calls. If not set
            defaults to us-central-1. Defaults to None.
        credentials (Optional[Credentials], optional): The credentials to use when making API
            calls. Defaults to None.

Naveena C.
·
Thank you
👍1

RunLLM
APP
·
Phoenix does not natively support selecting LLM-as-judge models directly from GCP Model Garden or arbitrary GCP endpoints. However, Phoenix supports Google's Gemini models via the Vertex AI Python SDK, allowing you to use Gemini LLMs for evaluation by authenticating with gcloud and using the GeminiModel class in Phoenix Evals. There is an open feature request to support the new google-genai SDK, but it is not yet implemented. Direct support for arbitrary GCP Model Garden models or custom endpoints is not mentioned in the data sources—only Vertex AI Gemini is supported at this time (feat: Add support for Google's Gemini models via Vertex python sdk, feature request: support google-genai as an eval model).Would you like more detail on how to use Gemini models with Phoenix, or on possible workarounds for custom endpoints?Sources: - feat: Add support for Google's Gemini models via Vertex python sdk - feature request: support google-genai as an eval model
💡 Hint: Mention RunLLM in the thread for followups.
Naveena C.
·
Yes, possible workarounds for custom endpoints would be helpful since it will help meet privacy and security related compliances.
Naveena C.
·
This will help in running generations on the fly too for certain models.
Xander S.
·
Hey Naveena C., we don't natively support Vertex AI in our playground, although please feel free to file us an enhancement request!
Naveena C.
·
Hi Xander S.! I am not looking for playground code but want to setup batch evaluation jobs using LLM evaluators that are hosted within my GCP project.
Xander S.
·
Got it! You should be able to use our LiteLLM model for that.
Xander S.
·
Actually, it looks like we have a native Vertex model https://arize.com/docs/phoenix/evaluation/how-to-evals/evaluation-models#vertexai
Xander S.
·
Can you give that a go to see if it achieves your aim?
Naveena C.
·
I checked LiteLLM earlier but that will send the hits to the actual LLM provider, right? I want to use self-hosted models as evaluators.

Xander S.

Not sure if this answers the question, but LiteLLM will provide a wrapper around the VertexAI SDK. I expect it should be possible to use VertexAIModel for your purpose.

        project (str, optional): The default project to use when making API calls. Defaults to None.
        location (str, optional): The default location to use when making API calls. If not set
            defaults to us-central-1. Defaults to None.
        credentials (Optional[Credentials], optional): The credentials to use when making API
            calls. Defaults to None.

Naveena C.
·
Thank you
👍1