Improve /healthz Endpoint to Validate DB Connectivity

Ben M. · 2025-04-23T19:14:00.685Z

Hey Arize team! We encountered an error today that tripped us up for a while. Our postgres connection was failing however the /healthz endpoint was returning OK, leading to us not properly reporting that the DB connection was the underlying issue. Would it be possible to verify DB connectivity on the healthz endpoint? Perhaps just run a select 1 against the database?

Anthony P.
·
Sorry to hear you were hung up on this for a while. Roger Y. or Dustin N. do you think there are any negative implications for something like this? On the one hand It may somewhat blur the line between service responsibilities but on the other you do need a database connection for phoenix to work properly.
Dustin N.
·
latency might be an issue, maybe we can make it an optional check or supplying a more comprehensive health check
Ben M.
·
Right now, the only thing that the current health endpoint checks is that python has not died, not that the app is functional, which, to my understanding, is what /healthz is typically meant for. I’d like to see it enabled by default due to the inability to have a working app without a database, but whichever direction it goes, having an option is crucial for us as we deploy more instances of phoenix 🙂
Roger Y.
·
we could add a readiness endpoint
Roger Y.
·
/readyz?
Ben M.
·
In previous roles deploying to k8s, we had 2 endpoints, one for readiness (Is initial startup done) and health (Is the app still functional)
Ben M.
·
These connections can fall apart after app startup, so teams I worked with made a point to test connections to our db in both of these so that our apps could be automatically bounced in an attempt to correct underlying issues. I’m curious what is driving the hesitation for a very light-weight call like select 1 in the hot path here.

Roger Y.

ok. maybe something like

HTTP/1.1 200 OK
Content-Type: application/json

{
  "status": "ready",
  "checks": {
    "database": { "status": "up", "latency_ms": 12 }
  }
}

Ben M.
·
I think the most important piece is that the status code is not a 2xx status code when the database is down
Roger Y.
·
ok. maybe something like 503
Ben M.
·
A 503 would be very fitting 🙂
Roger Y.
·
i don’t see any issue adding it, unless the team thinks otherwise
❤️1
Anthony P.
·
Writing an issue, will post momentarily
Ben M.
·
I’m even happy to help contribute if the team doesn’t have the bandwidth for this
Anthony P.
·
https://github.com/Arize-ai/phoenix/issues/7259

39 comments