Eugene E.

FYI, we're probably going to go with a managed solution anyway from you guys so my attention to the self-hosting issue might be limited

is that also true for gRPC, or you see no spans with gRPC?

similar state with gRPC, HTTP is somewhat fewer dropped spans but they're similar my endpoint is set like this:

PHOENIX_COLLECTOR_ENDPOINT="http://ec2-***-***-***-***.***.compute.amazonaws.com:4317"

so I believe it shouldn't use secure channel.

i have tried using http/protobuf as the transport with the self-hosted install, i end up getting "remote closed connection" errors, this is why i switched to grpc in the first place is i thought maybe that would resolve it

fyi it looks like your hosted solution doesn't have this issue

OTEL_EXPORTER_OTLP_TIMEOUT=600
OTEL_EXPORTER_OTLP_TRACES_TIMEOUT=600
OTEL_BSP_MAX_QUEUE_SIZE=2048*64
OTEL_BSP_SCHEDULE_DELAY=10000
OTEL_BSP_MAX_EXPORT_BATCH_SIZE=2048

seems to have made it worse, i'm still seeing the DEADLINE_EXCEEDED errors in the local logs but i think fewer traces are arriving in the collector

let me try setting them to smth large

we do have a sidecar for the DD export

or do i need to change the args on the created objects?

can i change these with env vars?

OTEL_BSP_MAX_QUEUE_SIZE
OTEL_BSP_SCHEDULE_DELAY
OTEL_BSP_MAX_EXPORT_BATCH_SIZE

to clarify - i've turned off the other tracing and the problem is still happening