Hey Dmitry - sorry for the delay. Early morning here.
We do perform load testing and can definitely get you some numbers. Let me organize our benchmarks in a way that it's more easily consumable.
Benchmarks however are a bit tricky as they vary on client, server, infra, resources, payloads, and volume. Each dimension matters.
In general though let me highlight some of the ways you can make things "production" ready.
Make sure to turn on batch processing. The batch processor accepts spans, metrics, or logs and places them into batches. Batching helps better compress the data and reduce the number of outgoing connections required to transmit the data. This processor supports both size and time based batching.
Switch to gRPC to maximize payload compression
Make sure to have facilities to scale up memory, disk, and possibly horizontal scaling.
Turn on backups of your postgres instance.
There's also additional facilities such as the OTEL collector to offload data if needed.
Phoenix works by taking the OTLP payloads off the network as soon as possible and queueing them for ingestion and joins. Since Phoenix tries to stay light on infra - this is managed by an in-memory queue and so RAM can be bumped if this becomes a bottleneck.
Let me know if you need any more details and also happy to chat more directly. It's also worth noting that we have. Arize Ax which is available via SAAS or a managed on-prem instance that has infrastructure behind it to manage high volume payloads if you ever decide to massively scale up.
Hope that helps.
