Parallelize experiments: We have large datasets in Phoenix with > 15k samples. I noticed that running experiments with large datasets is very slow. The data samples are seemingly processed sequentially. Is there a way to parallelize the processing, especially when hitting model providers like OpenAI? Parallelizing experiments is also beneficial when testing open-source LLMs like Scout. Setting up test instances is expensive, and those instances, like H100s, can handle multiple requests at the same time. It would reduce prototyping costs drastically if the requests could be parallelized.
