Issues with Phoenix Upgrade: Resource Exhausted Warnings
Hello, we have been using phoenix for some time now. we recently had to upgrade the version of the package (and its dependencies accordingly). according to the documentation, we instantiate the instrumentation as follows:
from openinference.instrumentation.langchain import LangChainInstrumentor
from phoenix.otel import register
trace_provider = register()
instrumentor = LangChainInstrumentor()
instrumentor.instrument(trace_provider=trace_provider)however, now we get some warnings on every call that also seem to "delay" the call and also seem to make the tracing inconsistent. the warnings look like that:
WARNING:opentelemetry.exporter.otlp.proto.grpc.exporter:Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 1s.
2024-10-09 08:33:44,263 - opentelemetry.exporter.otlp.proto.grpc.exporter - WARNING - Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 1s.
WARNING:opentelemetry.exporter.otlp.proto.grpc.exporter:Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 2s.
2024-10-09 08:33:45,270 - opentelemetry.exporter.otlp.proto.grpc.exporter - WARNING - Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 2s.
WARNING:opentelemetry.exporter.otlp.proto.grpc.exporter:Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 4s.
2024-10-09 08:33:47,278 - opentelemetry.exporter.otlp.proto.grpc.exporter - WARNING - Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 4s.
WARNING:opentelemetry.exporter.otlp.proto.grpc.exporter:Transient error StatusCode.RESOURCE_EXHAUSTED encountered while exporting traces to localhost:4317, retrying in 8s.
2are we doing something wrong? many thanks in advance (RunLLM i edited my message :))
