Prasanna

Arize team. I started playing around with Arize for the last two days, and I must admit that the Platform is slick. Some of the most challenging questions we usually get are about measuring LLM/RAG quality (relevance, hallucination, toxicity, bias), for which I never had a good answer. Arize gives a good point of view on what to look for in the LLM quality dimensions.

Thanks, Mikyo. Passing the api_base in model_kwargs worked. Appreciate it! While I have you, do you know how I can pass authentication parameters to a secured endpoint? The api_base is a model served and secured by a token.

response = litellm.completion(
    model="ollama/llama2", 
    messages=messages, 
    api_base="https://mydomain.com/serving-endpoints/llama2-7b-chat-hf/invocations"
)

Does the LitteLLMModel python class support ollama/llama2? Getting the error (enclosed) and any thoughts?

Hi. Do you plan to support the Databricks model-serving as an LLM provider? I am attempting to use LLMLite to wrap Databricks model-serving unsuccessfully.