Anton A.

It’s possible, I would need to connect with our infra team to find out

Yeah, I’m able to call the proxy API with streaming enabled as in the example you link, but haven’t tried server sent events — unsure if that’s separate or the same thing

Hmm, I can’t find any documentation which indicates that, so probably not 🤔 We’re hosting the standard LiteLLM Docker images

Hmm, we only call LLMs via our own base URLs + proxy token so not sure

In 7.10.3 it doesn’t redirect to the login anymore, but it shows an error in the UI “Failed to get output” with Stream enabled. And in the console:

WebSocket connection to 'wss://arize-phoenix..../graphql' failed: 

Perhaps this is a LiteLLM api limitation, I’m not sure. The docker logs do not show any further hints

Added an issue for better LiteLLM model configuration https://github.com/Arize-ai/phoenix/issues/6255

It works when I turn off the stream toggle

I’m on 7.10.1

Separately — it would be great if there’s a way to store custom model provider configuration. Ideally in a JSON file that’s supplied to the docker build We have a lot of custom models set up in the LiteLLM proxy & having to configure them every time you refresh is practically a showstopper. I’ll make an issue for this

Hmm, in our docker setup without auth it seems like the playground “Run” submission isn’t supported. I’m consistently redirected to the login page: login?returnUrl=%2Fplayground Network tab shows that:

Is setting up auth a pre-requisite for using the playground?