Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix: Support for BYO LLMs in Task Evaluation and Evals | Arize AI Community

Phoenix Support

Phoenix Support

Phoenix: Support for BYO LLMs in Task Evaluation and Evals

·Oct 30, 2024 03:19 PM

Does Pheonix support bringing my own LLMs for task evaluation? Will I still be able to use out of the box evals with a BYO backend LLM?

6 comments

· Sorted by Oldest

RunLLM
APP
·
Yes, Phoenix supports bringing your own LLMs for task evaluation. You can implement a custom LLM class by inheriting from a base evaluation model, allowing you to integrate your own LLMs with Phoenix's evaluation features. This includes using out-of-the-box evals with a custom backend LLM, as long as the custom model is compatible with Phoenix's evaluation framework. For more details, you can refer to the Custom Task Evaluation documentation.Would you like more detail on how to implement this? Sources: - Custom Task Evaluation
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
Yes! As long as you can use one of our model wrappers from phoenix.evals, many custom LLMs are supported using our LiteLLM model wrapper
Amy
·
Are there any code examples on how I can integrate Phoenix tracer with custom RAG agent (an agent that does not use popular frameworks like LlamaIndex?
Dustin N.
·
do you mean with manual instrumentation? we have several example notebooks
Amy
·
any chance you can link me to one? preferably RAG?
Dustin N.
·
Amy I don't think we have a RAG-specific example but here's the link to our docs: https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans