Managing Data Scalability in Phoenix with Retention Policies

·May 08, 2025 12:27 AM

I wonder how scalable phoenix is given it's using postgres and tracing can create lots of data. It feels periodically removing old data from postgres would be key for sustainability. I see

In Phoenix 9.0 or greater you will be able to automatically purge traces from projects by configuring a retention policy. Retention policies can be either time based or trace count based. Stay tuned for the 9.0 release.

on https://docs.arize.com/phoenix/settings/data-retention, is there ETA for when 9.0 would be available? Before 9.0, what would be the recommended way to keep data size under control e.g.

having a cron job to remove older data periodically by calling some phoenix APIs?
Any guidance on which APIs to call for data removal?

7 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix 9.0 will introduce automatic retention policies (time-based or trace count-based) for purging old traces, but there is no public ETA for the 9.0 release yet—only that it is "coming soon" per the <https://docs.arize.com/phoenix/settings/data-retention#P%3A4|official documentation>. Before 9.0, the recommended approach is to periodically remove old data manually or via automation. You can do this by scheduling a job that calls the GraphQL mutation `RemoveProjectDataFormMutation` to delete traces older than a specified date. Example mutation and payload are provided in <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1723659577.754099|this Slack thread> and <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1726502470.879049|here>. Would you like a step-by-step example of how to set up such a cron job? Sources: - <https://docs.arize.com/phoenix/settings/data-retention#P%3A4|Phoenix Data Retention Docs> - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1723659577.754099|Slack: Deleting Traces and Spans> - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1726502470.879049|Slack: Auto Deletion Guidance>
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
Hi Zhuyi X. we hope in the next couple days!
👍1
💡1
Zhuyi X.
·
I see 9.0.1 released and see https://github.com/Arize-ai/phoenix/blob/main/CHANGELOG.md
db table for trace data retention policies (#6703)
is there any doc about migration and how to enable data retention policy yet?
Dustin N.
·
hi Zhuyi X. it's released but we don't have the documentation ready yet, it should be out today!
Dustin N.
·
actually Zhuyi X. we just merged in the docs, here's the retention policy page: https://docs.arize.com/phoenix/settings/data-retention
👍1
Zhuyi X.
·
the change log says there is db table change for trace data, do I need to do any manual db migration or it's all automatic and I just need to swap to use the latest docker image?
Dustin N.
·
It’s all automatic
1

Managing Data Scalability in Phoenix with Retention Policies | Arize AI Community