500 Errors After Upgrading Phoenix from v13.13.0 to v13.16.0 and Database Migration Issues

·Mar 23, 2026 06:28 PM·

hello, we've been running phoenix for a while now, trying to keep our systems as close to the current release as possible. this morning i upgraded our QA system from v13.13.0 to v13.16.0. after updating we are getting 500 errors when we do an HTTP GET against / . i've tried downgrading to v13.13.0 and we get the same error now. i wonder there was a db migration that happened which maybe put the system in a bad state? here's the exeception:

8 comments

· Sorted by Oldest

RunLLM
APP
·
·
Yes, Phoenix performs database migrations automatically when upgrading to a new major version, and these migrations can change the schema in ways that are not backward-compatible. If you upgrade and then downgrade, the database schema will not be automatically reverted, which can leave the system in a "dirty" or incompatible state and cause errors like 500s on endpoints such as `/`. Manual intervention is required to revert migrations if you need to downgrade, as Phoenix does not support automatic schema downgrades. You should use the Alembic CLI to revert partial migrations and reset the migration state before restarting Phoenix. See the official migration documentation for details and troubleshooting steps <https://arize.com/docs/phoenix/self-hosting/upgrade/migrations|(source)>. If you need step-by-step instructions or help interpreting your specific exception, let me know and I can provide more detail. Sources: - <https://arize.com/docs/phoenix/self-hosting/upgrade/migrations|https://arize.com/docs/phoenix/self-hosting/upgrade/migrations>
💡 Hint: Mention RunLLM in the thread for followups.
Mikyo
·
·
Hey Peter W. we only ship migrations in major version bumps. Let me take a look at the logs you sent real quick
Mikyo
·
·
sorry you're hitting this
Peter W.
·
·
thanks!
Mikyo
·
·
Looks like there was a 1.0 release of starlette over the weekend that had some non backwards compatible signatures. We'll get a hotfix version out ASAP. sorry for the discruption. https://github.com/Arize-ai/phoenix/pull/12332
Peter W.
·
·
ah ok yea that makes sense. appreciate you looking into this
1
Peter W.
·
·
fantastic, i can confirm .17 fixes the issue we were running into 🙏
Mikyo
·
·
awesome, thanks for being the canary in the coal mine
💯1