Troubleshooting Could Not Parse Error in LLM Evaluations
Hi everyone, I'm getting a "Could not parse XXX" error when using a different LLM in the evaluations for the RAG example notebook: link. The issue occurs when I replace the example model with another one. Any tips on what might be causing this error? Please see the screenshot for error message details. Here's the modified code. Thank you!
# Original evaluators
# qa_evaluator = QAEvaluator(OpenAIModel(model="gpt-4-turbo-preview"))
# hallucination_evaluator = HallucinationEvaluator(OpenAIModel(model="gpt-4-turbo-preview"))
# Modified evaluators
qa_evaluator = QAEvaluator(BedrockModel(
# model_id="anthropic.claude-3-sonnet-20240229-v1:0",
client=bedrock_client))
hallucination_evaluator = HallucinationEvaluator(BedrockModel(
# model_id="meta.llama2-70b-chat-v1",
client=bedrock_client))
qa_correctness_eval_df, hallucination_eval_df = run_evals(
evaluators=[qa_evaluator, hallucination_evaluator],
dataframe=qa_with_reference_df,
provide_explanation=True,
concurrency=20,
verbose=True)