How to Fix Errors on /projects Page with 4TB Phoenix Database When GraphQL Loads Quickly

·Jun 11, 2026 02:47 PM·

Hi, I have 4TB database in phoenix, and receive error, when I load /projects page . But graphql request with same data works fast less 500 ms. How avoid this error ?

14 comments

· Sorted by Oldest

RunLLM
APP
·
(edited)·
If your /projects page is slow or errors out with a large Phoenix database (e.g., 4TB), but direct GraphQL queries are fast, the issue is likely due to the UI loading or aggregating too much data at once, or hitting timeouts or memory limits. This is a known issue—Phoenix's /projects page can block on metric queries for each project, causing slow loads or timeouts, especially with large datasets. Improvements have been made (e.g., lazy loading, reducing page size, and optimizing queries), but performance can still degrade with very large data volumes or many projects/traces. Upgrading to the latest Phoenix version is recommended, as recent releases include major speed improvements and better lazy loading for project views. Also, try reducing the number of projects, limiting the time range, or filtering to root spans to improve performance. If you still hit errors, check server logs for timeouts or memory errors, and consider increasing server resources or database tuning (e.g., indexes, partitioning) for large-scale deployments. For persistent issues, you may need to pre-aggregate metrics or further optimize your database setup (<https://github.com/Arize-ai/phoenix/issues/6868|source>, <https://github.com/Arize-ai/phoenix/issues/6201|source>, <https://arize.com/docs/phoenix/production-guide#memory-sizing|source>, <https://arize.com/docs/phoenix/self-hosting/architecture|source>, <https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements|source>). Would you like more detail on specific optimizations, configuration, or troubleshooting steps? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201> - <https://arize.com/docs/phoenix/production-guide#memory-sizing|https://arize.com/docs/phoenix/production-guide#memory-sizing> - <https://arize.com/docs/phoenix/self-hosting/architecture|https://arize.com/docs/phoenix/self-hosting/architecture> - <https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements|https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements>
💡 Hint: Mention RunLLM in the thread for followups.
Andrei L.
·
·
Have you specific github issue for this case ?
Andrei L.
·
·
RunLLM Have you specific github issue for this case ?
RunLLM
APP
·
(edited)
Yes, there are specific GitHub issues documenting this problem. The most relevant are <https://github.com/Arize-ai/phoenix/issues/6868|#6868> ("Blocking projects page metrics"—/projects page blocks on metric queries and can take up to 30 seconds or more to load with large data) and <https://github.com/Arize-ai/phoenix/issues/6201|#6201> ("Performance degradation when opening a project in Phoenix as the number of traces grows"—project page slows down as data increases, even with pagination). Both discuss root causes and ongoing improvements for large-scale Phoenix deployments. Would you like a summary of the solutions or workarounds discussed in these issues? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201>
💡 Hint: Mention RunLLM in the thread for followups.
Andrei L.
·
·
Both isses are close,
Andrei L.
·
·
RunLLM Both isses are close,
RunLLM
APP
·
(edited)·
Correct, both <https://github.com/Arize-ai/phoenix/issues/6868|#6868> and <https://github.com/Arize-ai/phoenix/issues/6201|#6201> are closed. However, the underlying performance issues with the /projects page on large datasets are acknowledged as ongoing challenges, and improvements have been released in newer Phoenix versions. If you are still experiencing problems, the maintainers recommend upgrading to the latest version and opening a new issue with detailed logs if the problem persists. Would you like guidance on what information to include if you open a new issue, or details on recent performance improvements? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201>
💡 Hint: Mention RunLLM in the thread for followups.
Roger Y.
·
·
Hi Andrei. Is this happening only intermittently?
Andrei L.
·
·
Yes, in average 50-70% times.
Andrei L.
·
·
🔒[private user] 🔒[private user] Beliz S. Thanks
Andrei L.
·
·
When huge load present, almost every time.
Roger Y.
·
·
graphql request with same data works fast less 500 ms
How did you measure this? Were you looking at it in the network tab of the browser dev tools?
When huge load present, almost every time.
Do you see any errors in the network tab when the UI errors out?
Roger Y.
·
·
there are two separate queries for the projects page as shown in the screenshot above. It could be that one of them is timing out
Andrei L.
·
·
I have create PR, to fix this issue, please take a look. https://github.com/Arize-ai/phoenix/pull/13748

How to Fix Errors on /projects Page with 4TB Phoenix Database When GraphQL Loads Quickly | Arize AI Community

RunLLM
APP
·
(edited)·
If your /projects page is slow or errors out with a large Phoenix database (e.g., 4TB), but direct GraphQL queries are fast, the issue is likely due to the UI loading or aggregating too much data at once, or hitting timeouts or memory limits. This is a known issue—Phoenix's /projects page can block on metric queries for each project, causing slow loads or timeouts, especially with large datasets. Improvements have been made (e.g., lazy loading, reducing page size, and optimizing queries), but performance can still degrade with very large data volumes or many projects/traces. Upgrading to the latest Phoenix version is recommended, as recent releases include major speed improvements and better lazy loading for project views. Also, try reducing the number of projects, limiting the time range, or filtering to root spans to improve performance. If you still hit errors, check server logs for timeouts or memory errors, and consider increasing server resources or database tuning (e.g., indexes, partitioning) for large-scale deployments. For persistent issues, you may need to pre-aggregate metrics or further optimize your database setup (<https://github.com/Arize-ai/phoenix/issues/6868|source>, <https://github.com/Arize-ai/phoenix/issues/6201|source>, <https://arize.com/docs/phoenix/production-guide#memory-sizing|source>, <https://arize.com/docs/phoenix/self-hosting/architecture|source>, <https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements|source>). Would you like more detail on specific optimizations, configuration, or troubleshooting steps? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201> - <https://arize.com/docs/phoenix/production-guide#memory-sizing|https://arize.com/docs/phoenix/production-guide#memory-sizing> - <https://arize.com/docs/phoenix/self-hosting/architecture|https://arize.com/docs/phoenix/self-hosting/architecture> - <https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements|https://arize.com/docs/phoenix/release-notes/03-2025/03-06-2025-project-improvements>
💡 Hint: Mention RunLLM in the thread for followups.
Andrei L.
·
·
Have you specific github issue for this case ?
Andrei L.
·
·
RunLLM Have you specific github issue for this case ?
RunLLM
APP
·
(edited)
Yes, there are specific GitHub issues documenting this problem. The most relevant are <https://github.com/Arize-ai/phoenix/issues/6868|#6868> ("Blocking projects page metrics"—/projects page blocks on metric queries and can take up to 30 seconds or more to load with large data) and <https://github.com/Arize-ai/phoenix/issues/6201|#6201> ("Performance degradation when opening a project in Phoenix as the number of traces grows"—project page slows down as data increases, even with pagination). Both discuss root causes and ongoing improvements for large-scale Phoenix deployments. Would you like a summary of the solutions or workarounds discussed in these issues? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201>
💡 Hint: Mention RunLLM in the thread for followups.
Andrei L.
·
·
Both isses are close,
Andrei L.
·
·
RunLLM Both isses are close,
RunLLM
APP
·
(edited)·
Correct, both <https://github.com/Arize-ai/phoenix/issues/6868|#6868> and <https://github.com/Arize-ai/phoenix/issues/6201|#6201> are closed. However, the underlying performance issues with the /projects page on large datasets are acknowledged as ongoing challenges, and improvements have been released in newer Phoenix versions. If you are still experiencing problems, the maintainers recommend upgrading to the latest version and opening a new issue with detailed logs if the problem persists. Would you like guidance on what information to include if you open a new issue, or details on recent performance improvements? Sources: - <https://github.com/Arize-ai/phoenix/issues/6868|https://github.com/Arize-ai/phoenix/issues/6868> - <https://github.com/Arize-ai/phoenix/issues/6201|https://github.com/Arize-ai/phoenix/issues/6201>
💡 Hint: Mention RunLLM in the thread for followups.
Roger Y.
·
·
Hi Andrei. Is this happening only intermittently?
Andrei L.
·
·
Yes, in average 50-70% times.
Andrei L.
·
·
🔒[private user] 🔒[private user] Beliz S. Thanks
Andrei L.
·
·
When huge load present, almost every time.
Roger Y.
·
·
graphql request with same data works fast less 500 ms
How did you measure this? Were you looking at it in the network tab of the browser dev tools?
When huge load present, almost every time.
Do you see any errors in the network tab when the UI errors out?
Roger Y.
·
·
there are two separate queries for the projects page as shown in the screenshot above. It could be that one of them is timing out
Andrei L.
·
·
I have create PR, to fix this issue, please take a look. https://github.com/Arize-ai/phoenix/pull/13748