Managing Data Scalability in Phoenix with Retention Policies

·May 08, 2025 12:27 AM

I wonder how scalable phoenix is given it's using postgres and tracing can create lots of data. It feels periodically removing old data from postgres would be key for sustainability. I see

In Phoenix 9.0 or greater you will be able to automatically purge traces from projects by configuring a retention policy. Retention policies can be either time based or trace count based. Stay tuned for the 9.0 release.

on https://docs.arize.com/phoenix/settings/data-retention, is there ETA for when 9.0 would be available? Before 9.0, what would be the recommended way to keep data size under control e.g.

having a cron job to remove older data periodically by calling some phoenix APIs?
Any guidance on which APIs to call for data removal?

7 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix 9.0 will introduce automatic retention policies (time-based or trace count-based) for purging old traces, but there is no public ETA for the 9.0 release yet—only that it is "coming soon" per the official documentation.Before 9.0, the recommended approach is to periodically remove old data manually or via automation. You can do this by scheduling a job that calls the GraphQL mutation RemoveProjectDataFormMutation to delete traces older than a specified date. Example mutation and payload are provided in this Slack thread and here. Would you like a step-by-step example of how to set up such a cron job?Sources: - Phoenix Data Retention Docs - Slack: Deleting Traces and Spans - Slack: Auto Deletion Guidance
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
Hi Zhuyi X. we hope in the next couple days!
👍1
💡1
Zhuyi X.
·
I see 9.0.1 released and see https://github.com/Arize-ai/phoenix/blob/main/CHANGELOG.md
db table for trace data retention policies (#6703)
is there any doc about migration and how to enable data retention policy yet?
Dustin N.
·
hi Zhuyi X. it's released but we don't have the documentation ready yet, it should be out today!
Dustin N.
·
actually Zhuyi X. we just merged in the docs, here's the retention policy page: https://docs.arize.com/phoenix/settings/data-retention
👍1
Zhuyi X.
·
the change log says there is db table change for trace data, do I need to do any manual db migration or it's all automatic and I just need to swap to use the latest docker image?
Dustin N.
·
It’s all automatic
1