hey everyone! I'm trying to create an eval - task but I'm getting this error:
Error creating task Relay request for CreateEvalTaskMutation failed by the following reasons: 1. OpenAI requires a modelName createEvalTask(input: $input) { ^^^I just added OpenAI's as provider and added a custom model 'gpt-4o', but still doesn't work. Any ideas what setup I'm missing?
where's the llmconfig block on arize ax?
Hi Ezequiel R. - taking a look! Can you try removing the custom model 'gpt-4o'. Or delete it and configure again without touching the advanced settings. The reason being is that when you configure OpenAI, you should already be able to have access to gpt-4o. I just tested this out on my end and I was able to create the eval task. If this wasn't the fix, let me know
I was able to get it working by entering the custom model the way you did too. So it is possible that the solution above is not the fix. Let me know and we can dig deeper
Thank you for your time and response 馃敀[private user]. I re-create the openAI integration as you mentioned but I see the same error. Not sure what's going on. Just in case, I used this guide to get spans from my nextjs app: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-tracing Besides that I didn't touch anything on Arize AI.... Any other ideas?
We are looking into it on our end! Meanwhile, can you show me what you are entering when you click LLM parameters on the "New LLM-As-A-Judge Eval" page Ezequiel R.
That was the missing thing! I didn't even see it. I was able to create it. Thanks!! (I'll recommend add it to the steps or paint in red when the user clicks on create evan and it wasn't selected)
Ah glad it works! Good to know - we note to improve it in docs/UI flow.
