KeyError Issue with Relevance Evaluator in GCP Environment
Hi Team, I have a question, hope you can help me with it. I have a working environment on my machine where I can successfully run relevance evaluators using GPT4o. Here is the code:
relevance_evaluator = RelevanceEvaluator(OpenAIModel(model="gpt-4o"))
retrieved_docs_with_relevance_df = run_evals(
evaluators=[relevance_evaluator],
dataframe=root_retrieved_df,
provide_explanation=True,
concurrency=20,
)[0]
run_evals |██████████| 9/9 (100.0%) | ⏳ 00:05<00:00 | 1.66it/sI am running the exact same thing on another machine running in GCP:
relevance_evaluator = RelevanceEvaluator(
OpenAIModel(model="gpt-4o")
)
retrieved_docs_with_relevance_df = run_evals(
evaluators=[relevance_evaluator],
dataframe=root_retrieved_df,
provide_explanation=True,
concurrency=20,
)[0]But I'm getting this weird error:
Exception in worker on attempt 1: raised KeyError('reference')
Requeuing...
Exception in worker on attempt 1: raised KeyError('reference')
Requeuing...
Exception in worker on attempt 1: raised KeyError('reference')
Requeuing...
Exception in worker on attempt 1: raised KeyError('reference')
Requeuing...
Exception in worker on attempt 1: raised KeyError('reference')
Requeuing...I have the following versions in GCP: arize-phoenix==4.5.0 , arize-phoenix-evals==0.13.0 And the following versions on my laptop: arize-phoenix==3.16.3 and using the experimental eval imports:
from phoenix.experimental.evals import (
RelevanceEvaluator,
run_evals,
)I suspect that this is related to the versions probably, I was wondering if you know what the problem is?
