Switch to GPT-4 Turbo for Faster, Cheaper Evals Performance

·Nov 14, 2023 05:09 PM

GPT-4 Turbo Use on Evals: We put together our perspective on GPT-4 Turbo with a ton of research and testing. TLDR: We are recommending most customers move to GPT-4 Turbo for Evals (if on GPT-4 or 3.5). It's 2x as fast as GPT-4 with similar performance across our Evals set. Awesome model performance that beats everything else on market in performance AND is cheaper and faster.

🚄3

Discussions

Switch to GPT-4 Turbo for Faster, Cheaper Evals Performance

Jason

·Nov 14, 2023 05:09 PM

GPT-4 Turbo Use on Evals: We put together our perspective on GPT-4 Turbo with a ton of research and testing. TLDR: We are recommending most customers move to GPT-4 Turbo for Evals (if on GPT-4 or 3.5). It's 2x as fast as GPT-4 with similar performance across our Evals set. Awesome model performance that beats everything else on market in performance AND is cheaper and faster.

🚄3

2 comments

· Sorted by Oldest

Christos
·
What eval framework is this? Curious how to replicate.
Jason
·
Christos Its from our Phoenix Open Source team - Phoenix has an Eval library which includes the golden test datasets and easy integrations with these models. The notebooks are included for each Eval and is pretty easy to run for OpenAI. We can drop the snipits of code to instantiate the other models if you have interest. https://docs.arize.com/phoenix/llm-evals/llm-evals Example: https://docs.arize.com/phoenix/llm-evals/running-pre-tested-evals/retrieval-rag-relevance
🤝1
🙏1

Christos
·
What eval framework is this? Curious how to replicate.
Jason
·
Christos Its from our Phoenix Open Source team - Phoenix has an Eval library which includes the golden test datasets and easy integrations with these models. The notebooks are included for each Eval and is pretty easy to run for OpenAI. We can drop the snipits of code to instantiate the other models if you have interest. https://docs.arize.com/phoenix/llm-evals/llm-evals Example: https://docs.arize.com/phoenix/llm-evals/running-pre-tested-evals/retrieval-rag-relevance
🤝1
🙏1