Elizabeth H.

Nico G., have you tried increasing the timeout from the default?

In the typescript library, we delegate to the AI SDK from Vercel, but there isn't one clear "winner" on the python side.

You're the first person we've heard of using this llm tool. We decided to start with adaptors to the 3 most common LLM clients we see people using: openai, langchain, and litellm. Between those 3, pretty much every model is supported. We intentionally avoided strict dependency on any one package, such that users can choose which SDK they prefer to use.

Just a heads up, though, that we are still working on supporting LLM invocation parameters with evals 2.0 so you won't be able to specify max_tokens or max_completion_tokens right now. We are actively working on this now so it should be available soon. cc: 🔒[private user], 🔒[private user]

We are also working on the docs for the LLM abstraction. We limited provider support to anthropic and openai initially because we haven't tested others, but we can theoretically expand to support any provider compatible with one of openai, langchain, or litellm as client SDKs. There are two places where you can pass additional kwargs - at LLM instantiation you should pass any "client" args like api_base . You can pass invocation params like temperature to the LLM methods, but we haven't set up the LLMEvaluator/ClassificationEvaluator to pass through kwargs yet. We are getting to it soon.

Rasmus B. thank you for your patience as we update our docs! [This notebook](https://github.com/Arize-ai/phoenix/blob/main/tutorials/evals/evals-2/evals_2.0_rag_demo.ipynb) has examples of how to use evals-2 in experiments - all evals evaluators have drop-in compatibility as experiment evaluators. Hopefully this is helpful in the meantime