Inquiring About Evaluator Class for HUMAN_VS_AI in Phoenix

·Feb 24, 2024 04:27 AM

Hi There, Does Phoenix have builtin Evaluator class for HUMAN_VS_AI , I see HUMAN_VS_AI_PROMPT_TEMPLATE, but didn't find evaluator by searching "human", also didn't find in /experimental/evals/evaluators.py Not sure if I miss something. Thanks!

Phoenix Support

Inquiring About Evaluator Class for HUMAN_VS_AI in Phoenix

xuejiao w.

·Feb 24, 2024 04:27 AM

Hi There, Does Phoenix have builtin Evaluator class for HUMAN_VS_AI , I see HUMAN_VS_AI_PROMPT_TEMPLATE, but didn't find evaluator by searching "human", also didn't find in /experimental/evals/evaluators.py Not sure if I miss something. Thanks!

3 comments

· Sorted by Oldest

Mikyo
·
Hi xuejiao w., there is no evaluator for it but you can run it on your results if you have ground truth human results! https://docs.arize.com/phoenix/evaluation/running-pre-tested-evals/ai-vs-human-groundtruth
Junaid K.
·
Can we integrate with Argilla ?
xuejiao w.
·
Mikyo Got it . Thanks!
🙌1

Mikyo
·
Hi xuejiao w., there is no evaluator for it but you can run it on your results if you have ground truth human results! https://docs.arize.com/phoenix/evaluation/running-pre-tested-evals/ai-vs-human-groundtruth
Junaid K.
·
Can we integrate with Argilla ?
xuejiao w.
·
Mikyo Got it . Thanks!
🙌1