To achieve tracing across multiple services deployed in separate containers, you need to propagate the trace context (trace ID and span ID) across service boundaries. This involves passing the trace context from the main backend service to the moderation service and the search + answer service via HTTP headers.
Here's a step-by-step approach:
1. Main Backend Service: Start a top-level span and propagate the trace context in the HTTP headers when making REST API calls to other services.
2. Moderation Service and Search + Answer Service: Extract the trace context from the incoming HTTP headers and continue the trace.
Main Backend Service
```
from opentelemetry import trace
from opentelemetry.propagate import inject
tracer = trace.get_tracer(name)
with tracer.start_as_current_span("top_level_trace") as span:
span.set_attribute("openinference.span.kind", "CHAIN")
# Prepare headers for propagation
headers = {}
inject(headers)
# Call moderation service
response = [requests.post](http://requests.post)("[http://moderation-service/api](http://moderation-service/api)", headers=headers, json=payload)
# Call search + answer service
response = [requests.post](http://requests.post)("[http://search-answer-service/api](http://search-answer-service/api)", headers=headers, json=payload)
LBKS**Moderation Service and Search + Answer Service**
LBKS```
from opentelemetry import trace
from opentelemetry.propagate import extract
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
# Setup tracer provider
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint="[http://phoenix:6006/v1/traces](http://phoenix:6006/v1/traces)")))
trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)
# Extract trace context from incoming request headers
def handle_request(request):
context = extract(request.headers)
with tracer.start_as_current_span("service_span", context=context) as span:
span.set_attribute("openinference.span.kind", "CHAIN")
# Service logic here
pass
This approach ensures that all spans are grouped under a single top-level trace, even when the services are deployed in separate containers.
Would you like more detail on any specific part?
Sources:
- Slack Discussion
