Need Help Configuring a Custom API Endpoint in Prompt Playground?

hugo b. · 2025-04-26T23:39:32.440Z

hey all! Prompt Playground looks great — I need to compare different prompts quickly. I have a simple API that accepts a prompt and returns a completion. From the UI, it doesn’t look like I can configure a custom API endpoint — am I missing something? (For example: if I stood up a basic endpoint on Modal, Baseten, etc., could I connect that?) Thanks!

Need Help Configuring a Custom API Endpoint in Prompt Playground? | Arize AI Community

12 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix's Prompt Playground supports various AI providers and custom model endpoints, allowing you to experiment with and evaluate prompts and models for your tasks. However, the data sources do not explicitly mention the ability to configure a custom API endpoint directly through the UI for the Prompt Playground. To connect a custom API endpoint, you might need to use manual instrumentation or custom spans as described in the Phoenix documentation. This approach allows you to trace requests and add evaluations to the traces captured in Phoenix, which could be adapted for your custom API.Would you like more detail on how to set this up? Sources: - Prompt Playground
💡 Hint: Mention RunLLM in the thread for followups.
Anthony P.
·
Glad to hear you are enjoying the playground! It is not yet possible unfortunately to configure custom providers, unless your new API is strictly compatible with the OpenAI client, in which case you can change the base url in the prompt playground model settings for OpenAI. This feature is on our roadmap however, and we plan to take it on within the next few features we release!
🙌1
hugo b.
·
i really appreciate the quick follow up, Tony, and really excited for the future here: it's still early days 🤗 just to confirm, the “Base URL” field under OpenAI settings (screenshot attached) is the right place to configure a custom endpoint, correct? Also, to double check: for OpenAI compatibility, I’m assuming you expect something like:
POST requests to /v1/chat/completions (or /v1/completions)
Standard request fields like model, messages (for chat), prompt (for completions), temperature, etc.
Standard response format with choices, usage, etc.
JSON headers and bearer auth
Is that the full set of expectations? Or is there anything else important I should align to? Thanks again for all the help!
🙏
Anthony P.
·
That is the full set of expectations yes, there are many LLM providers that adhere to the openai schema for chat completions such as Ollama, Gemini, and more. You may be able to leverage some kind of API starter that will serve an LLM with this schema if you want something more custom than Ollama
✅1
hugo b.
·
thanks, man. I've got a RAG+Agent system deployed on modal rn & ill play around with getting it serve this type of endpoint. pretty sure cursor + sonnet 3.7 or gemini 2.5 could probably spec it out and build it for me tbh 😜
hugo b.
·
(lol just realized i suggested to myself 2 LLMs that arent from OAI: but they're the best atm imo!)
Anthony P.
·
They are my favorite as well 😂 are you planning a custom model or something off the shelf? It's likely that a proxy of some kind may exist already
Anthony P.
·
Something like the self hostable lite LLM proxy server may interest you if they support a model you intend to use https://docs.litellm.ai/docs/proxy/deploy
hugo b.
·
thanks! its a custom Agentic RAG but using a vendor API for the LLM calls yeah i like litellm id prefer to see how i can hack what ive already served up on modal to also serve an endpoint but this may be the way in the end ^
Anthony P.
·
I admire the hacker spirit 🫡 good luck!
hugo b.
·
great time to hack away!
hugo b.
·
actually it may be the most amazing time our civilization has ever seen so far to hack away?! 🤖 🚀 🤗
1

RunLLM
APP
·
Phoenix's Prompt Playground supports various AI providers and custom model endpoints, allowing you to experiment with and evaluate prompts and models for your tasks. However, the data sources do not explicitly mention the ability to configure a custom API endpoint directly through the UI for the Prompt Playground. To connect a custom API endpoint, you might need to use manual instrumentation or custom spans as described in the Phoenix documentation. This approach allows you to trace requests and add evaluations to the traces captured in Phoenix, which could be adapted for your custom API.Would you like more detail on how to set this up? Sources: - Prompt Playground
💡 Hint: Mention RunLLM in the thread for followups.
Anthony P.
·
Glad to hear you are enjoying the playground! It is not yet possible unfortunately to configure custom providers, unless your new API is strictly compatible with the OpenAI client, in which case you can change the base url in the prompt playground model settings for OpenAI. This feature is on our roadmap however, and we plan to take it on within the next few features we release!
🙌1
hugo b.
·
i really appreciate the quick follow up, Tony, and really excited for the future here: it's still early days 🤗 just to confirm, the “Base URL” field under OpenAI settings (screenshot attached) is the right place to configure a custom endpoint, correct? Also, to double check: for OpenAI compatibility, I’m assuming you expect something like:
POST requests to /v1/chat/completions (or /v1/completions)
Standard request fields like model, messages (for chat), prompt (for completions), temperature, etc.
Standard response format with choices, usage, etc.
JSON headers and bearer auth
Is that the full set of expectations? Or is there anything else important I should align to? Thanks again for all the help!
🙏
Anthony P.
·
That is the full set of expectations yes, there are many LLM providers that adhere to the openai schema for chat completions such as Ollama, Gemini, and more. You may be able to leverage some kind of API starter that will serve an LLM with this schema if you want something more custom than Ollama
✅1
hugo b.
·
thanks, man. I've got a RAG+Agent system deployed on modal rn & ill play around with getting it serve this type of endpoint. pretty sure cursor + sonnet 3.7 or gemini 2.5 could probably spec it out and build it for me tbh 😜
hugo b.
·
(lol just realized i suggested to myself 2 LLMs that arent from OAI: but they're the best atm imo!)
Anthony P.
·
They are my favorite as well 😂 are you planning a custom model or something off the shelf? It's likely that a proxy of some kind may exist already
Anthony P.
·
Something like the self hostable lite LLM proxy server may interest you if they support a model you intend to use https://docs.litellm.ai/docs/proxy/deploy
hugo b.
·
thanks! its a custom Agentic RAG but using a vendor API for the LLM calls yeah i like litellm id prefer to see how i can hack what ive already served up on modal to also serve an endpoint but this may be the way in the end ^
Anthony P.
·
I admire the hacker spirit 🫡 good luck!
hugo b.
·
great time to hack away!
hugo b.
·
actually it may be the most amazing time our civilization has ever seen so far to hack away?! 🤖 🚀 🤗
1