How to Run Experiments in Batch for Cost-Efficient LLM Evaluation
Hi Team May I ask about how to run experiment by batch Normally, we run LLMs with real-time, anyway for experimentation evaluation we can use batch api with 50% lower costs. https://platform.openai.com/docs/guides/batch Example
Run batch from dataset to save output in Phoenix first, then tun evaluation by batch after that
Please feel free to share, thanks in advance.