Hello! I'm integrating Arize into an Elixir application that already uses OpenTelemetry to trace web requests, DB queries, background jobs, and almost everything else. So I have 2 OpenTelemetry exporters in my app. Which means Arize is getting all of the traces (web, db, ...). Is there a way to tell Arize to ignore those?? I know I could set up a local collector and use a span filter, but I don't want to spin up a new Docker container for this. for reference, my config:
resource_attributes = %{
"service.name" => "specialist",
"service.version" => app_version,
"deployment.environment" => env!("MIX_ENV", :string, "dev"),
"model_id" => arize_project_name
}
# Arize headers
arize_headers = [
{"api_key", arize_api_key},
{"space_id", arize_space_key},
{"model_id", arize_project_name}
]
config :opentelemetry,
resource: resource_attributes,
sampler: {:otel_sampler_always_on, %{}},
resource_detectors: [:otel_resource_env_var, :otel_resource_app_env],
text_map_propagators: [:baggage, :trace_context],
processors: [
otel_batch_processor: %{
name: :company_processor,
exporter: {
:opentelemetry_exporter,
%{
protocol: :http_protobuf,
endpoints: [company_endpoint]
}
}
},
otel_batch_processor: %{
name: :arize_processor,
exporter: {
:opentelemetry_exporter,
%{
protocol: :http_protobuf,
endpoints: ["https://otlp.arize.com"],
headers: arize_headers
}
}
}
]
🔒[private user] do you have a collector config.yaml example that works with Arize? Mine is not working (and I configured model_id, api_key, space_id)
ot-collector-1 | 2025-05-08T14:07:31.645Z debug otlphttpexporter@v0.116.0/otlp.go:177 Preparing to make HTTP request {"kind": "exporter", "data_type": "traces", "name": "otlphttp/arize", "url": "https://otlp.arize.com/v1/traces"}
ot-collector-1 | 2025-05-08T14:07:31.795Z error internal/queue_sender.go:103 Exporting failed. Dropping data. {"kind": "exporter", "data_type": "traces", "name": "otlphttp/arize", "error": "not retryable error: Permanent error: rpc error: code = InvalidArgument desc = error exporting items, request to https://otlp.arize.com/v1/traces responded with HTTP Status Code 400, Message=proto: cannot parse invalid wire-format data, Details=[]", "dropped_items": 46}
tks
seems I got it. For reference, this is my collector config for development (sending to both Jaeger and Arize):
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
batch:
timeout: 1s
send_batch_size: 1024
resource:
attributes:
- key: service.name
value: "specialist"
action: upsert
- key: model_id
value: "specialist-dev"
action: upsert
- key: deployment.environment
value: "development"
action: upsert
# Send to Arize every span that starts with specialist.llm
filter:
error_mode: ignore
traces:
span:
- 'not IsMatch(name, "specialist.llm.*")'
exporters:
# Debug exporter for development troubleshooting
debug:
verbosity: detailed
# Jaeger exporter for local trace visualization
otlp/jaeger:
endpoint: jaeger:4317
tls:
insecure: true
# Arize exporter for LLM observability
otlp/arize:
endpoint: "otlp.arize.com:443"
tls:
insecure: false
headers:
api_key: ""
space_id: ""
model_id: "specialist-dev"
service:
telemetry:
logs:
level: "debug"
pipelines:
# Pipeline for all traces to Jaeger
traces/jaeger:
receivers: [otlp]
processors: [resource, batch]
exporters: [debug, otlp/jaeger]
# Dedicated pipeline for LLM traces to Arize
traces/arize:
receivers: [otlp]
processors: [resource, filter, batch]
exporters: [debug, otlp/arize]