it would be pretty useful to be able to upload or download evals from the SDK, like huggingface_hub
Great idea, we are actually looking into an evaluator hub! I'll check out the Huggingface Hub and will circle back once we have a more detailed roadmap to get your thoughts
Yeah that's correct, we are just calling llm_classify in Arize online evals. There are not any differences--the only other component is how we load in and pre-process the data before calling llm_classify, which is visible from the Test in Code button
Thanks for flagging! We will support the developer role and add better support for reasoning models in general by the end of next sprint (Feb 23) -- just catching up to the recent API changes from Open AI as you mentioned 🙂https://platform.openai.com/docs/guides/reasoning#advice-on-prompting
🎉 Video Deep Dives on Latest Releases🎉
Hey everyone, just dropping some videos that we put together which give a more detailed overview of some of the releases mentioned above
