Can Phoenix Handle Multiple Instances on GCP CloudRun?

Roméo I. · 2024-11-04T08:13:36.459Z

Hello All ! I have a question for you guys, my phoenix server is running on a GCP CloudRun service, currently with one and only one instance, my DB is a postgresdb. Does phoenix support multiple instances running concurrently to handle the load for a broad traffic ?

7 comments

· Sorted by Oldest

RunLLM
APP
·
Yes, Phoenix supports running multiple instances concurrently. According to Arize AI Slack, Phoenix is stateless, allowing you to scale out by deploying multiple Phoenix containers backed by a single Postgres database. This setup can help handle increased traffic and workload.Would you like more detail on how to configure this setup? Sources: - Arize AI Slack
💡 Hint: Mention RunLLM in the thread for followups.
Roméo I.
·
Awesome ! Thanks for the reply, Yes I would like more details please.
Xander S.
·
hey Roméo I., please see the self-hosted section of our documentation https://docs.arize.com/phoenix/deployment
Roméo I.
·
Thanks Xander, didn't saw the kubernetes section last time I checked, is it a new addition ?
Roméo I.
·
Regarding this subject, I made a stress test of my application yesterday, and I noticed the traces were not saved correctly when phoenix receive a big volumes at the same time. My phoenix servers are running on GCP Cloud run with 1vcpu and 2Go of Ram per instances and can grow up to 10 instances, during the test my maximum number of instances was 3. The image I used for my phoenix server is 4.33.2. Is this a known issue, what can I do to mitigate/resolve the probleme ?
Xander S.
·
Thanks for your patience Roméo I.
Thanks Xander, didn't saw the kubernetes section last time I checked, is it a new addition ?
Not new, but we may have reorganized.
Regarding this subject, I made a stress test of my application yesterday, and I noticed the traces were not saved correctly when phoenix receive a big volumes at the same time. My phoenix servers are running on GCP Cloud run with 1vcpu and 2Go of Ram per instances and can grow up to 10 instances, during the test my maximum number of instances was 3. The image I used for my phoenix server is 4.33.2. Is this a known issue, what can I do to mitigate/resolve the probleme ?
Can you file us an issue on GitHub with more details? What load Phoenix was under, what was the symptom of failure, etc.?
Roméo I.
·
Thanks Xander S. for your reply, will do ASAP 😉

RunLLM
APP
·
Yes, Phoenix supports running multiple instances concurrently. According to Arize AI Slack, Phoenix is stateless, allowing you to scale out by deploying multiple Phoenix containers backed by a single Postgres database. This setup can help handle increased traffic and workload.Would you like more detail on how to configure this setup? Sources: - Arize AI Slack
💡 Hint: Mention RunLLM in the thread for followups.
Roméo I.
·
Awesome ! Thanks for the reply, Yes I would like more details please.
Xander S.
·
hey Roméo I., please see the self-hosted section of our documentation https://docs.arize.com/phoenix/deployment
Roméo I.
·
Thanks Xander, didn't saw the kubernetes section last time I checked, is it a new addition ?
Roméo I.
·
Regarding this subject, I made a stress test of my application yesterday, and I noticed the traces were not saved correctly when phoenix receive a big volumes at the same time. My phoenix servers are running on GCP Cloud run with 1vcpu and 2Go of Ram per instances and can grow up to 10 instances, during the test my maximum number of instances was 3. The image I used for my phoenix server is 4.33.2. Is this a known issue, what can I do to mitigate/resolve the probleme ?
Xander S.
·
Thanks for your patience Roméo I.
Thanks Xander, didn't saw the kubernetes section last time I checked, is it a new addition ?
Not new, but we may have reorganized.
Regarding this subject, I made a stress test of my application yesterday, and I noticed the traces were not saved correctly when phoenix receive a big volumes at the same time. My phoenix servers are running on GCP Cloud run with 1vcpu and 2Go of Ram per instances and can grow up to 10 instances, during the test my maximum number of instances was 3. The image I used for my phoenix server is 4.33.2. Is this a known issue, what can I do to mitigate/resolve the probleme ?
Can you file us an issue on GitHub with more details? What load Phoenix was under, what was the symptom of failure, etc.?
Roméo I.
·
Thanks Xander S. for your reply, will do ASAP 😉