Troubleshooting Orphaned Spans in Google Cloud with OpenTelemetry | Arize AI Community

Arize AI Community Icon

Troubleshooting Orphaned Spans in Google Cloud with OpenTelemetry | Arize AI Community

Saurabh K.
·
·
yes , I want step by step process. Also I see that in spans Attributes I have user email and id. But in display I only see the id , the email column is empty. "user": { "email": "saurabh@obin.ai", "id": "user_33Sq6QIvzI0aLs4wpLtWNLV1lM3", "tenant": "default", "token_type": "clerk_jwt" },
Saurabh K.
·
·
1.
RunLLM see above screenshot and problem statement
Saurabh K.
·
·
attributes.user.id and attributes.user.email are at the same hierarchy 🔒[private user]
Dat
External
·
·
Saurabh K. just to confirm locally is good, but container is missing there might be a couple reasons for this, can you try a flush in your container, my gut is that you're batching export the spans and the spans might not be exciting the application
Dat
External
·
·
https://arize.com/docs/ax/observe/tracing/configure-tracing-options/customize-auto-instrumentation#batch-vs-simple-span-processors
Dat
External
·
·
use the OTEL flush and it might fix it
Dat
External
·
·
keep me posted if it works
Saurabh K.
·
·
where should I use the OTEL flush, am using python arize-otel. Also my containers are long running containers, not ephemeral ?
Saurabh K.
·
In the local and GCP container, I want to use the same. So BatchSpanProcessor/SimpleSpanProcessor should be same. How do I verify which one is on mylocal ?
Dat
External
·
·
Saurabh K. just to make sure all the spans are in arize, they are just coming in separately? can you screenshot the local traces (what you want) vs what happening in container (what you don't want)
Saurabh K.
·
·
🔒[private user] So on the gcloud container instances, my Traces tab is empty but my Spans tab are getting filled. The Caution ! symbol, says that it's Orphan span, parent span not found in Traces
Saurabh K.
·
But on the local python run, everything works great
Dat
External
·
·
can you show me the local traces
Dat
External
·
·
just for me to reference
Saurabh K.
·