Yes, Phoenix does implement retry mechanisms when running prompts in bulk, particularly in scenarios where rate limiting might be encountered. According to the <https://github.com/Arize-ai/phoenix/issues/1784|Phoenix GitHub repository>, Phoenix has enhanced its evaluation verbose mode to log retries and failed requests, which helps users understand the cause of any slowdown due to rate limits. Additionally, the <https://github.com/Arize-ai/phoenix/blob/main/packages/phoenix-evals/src/phoenix/evals/models/rate_limiters.py|rate limiters.py file> in the Phoenix codebase describes an adaptive rate limiter that adjusts the request rate based on the number of rate limit errors, reducing the rate when errors occur and gradually increasing it over time if no further errors are encountered.
Would you like more detail on how the retry mechanism is implemented or how to configure it?
Sources:
- <https://github.com/Arize-ai/phoenix/issues/1784|Phoenix GitHub repository>
- <https://github.com/Arize-ai/phoenix/blob/main/packages/phoenix-evals/src/phoenix/evals/models/rate_limiters.py|rate limiters.py file>
