Logging Custom Metrics in Spans for Dashboard Visibility

·Mar 18, 2025 08:49 PM

I have a doubt regarding logging our own metrics per span. I see annotations is a way, but it's not clear on how I can log my own metric within a span (for it to appear on the dashboard in the feedback column.

21 comments

· Sorted by Oldest

RunLLM
APP
·
To log your own metrics per span and have them appear in the feedback column on the Phoenix dashboard, you can use annotations. Annotations allow you to attach custom metrics to spans. Here's a concise breakdown: 1. *Create Annotations*: Use the Phoenix API to create annotations for your spans. An annotation can include a label, score, and explanation, which can represent your custom metric. 2. *Log Annotations*: Send these annotations to Phoenix using the `/v1/span_annotations` endpoint. Ensure that each annotation is associated with the correct `span_id`. For example, you can create an annotation with a custom metric like this: ``` import httpx client = httpx.Client() annotation_payload = { "data": [ { "span_id": "your_span_id", "name": "custom_metric", "annotator_kind": "HUMAN", "result": {"label": "metric_label", "score": 1, "explanation": "Metric explanation"}, "metadata": {}, } ] } headers = {'api_key': '<your phoenix api key> '} client.post( "https://app.phoenix.arize.com/v1/span_annotations?sync=false", json=annotation_payload, headers=headers ) ``` This will log your custom metric as an annotation, which will then appear in the feedback column on the Phoenix dashboard. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback> - <https://docs.arize.com/phoenix/tracing/features-tracing/how-to-annotate-traces#S%3A3|How to Annotate Traces>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM what if I want to do in real-time? I have a method where I've defined a span, it's attributes etc. That's where i get the value of the metric, and I want to log it from there like span.set_attribute(), without having to do this client.post()
RunLLM
APP
·
To log metrics in real-time directly within a span, you can use the `set_attribute` method on the span to attach your custom metric as an attribute. This approach allows you to add the metric value directly to the span without making an additional HTTP request. Here's how you can do it: 1. *Set Attribute on Span*: Use the `set_attribute` method to add your custom metric as an attribute to the span. This will ensure that the metric is logged as part of the span's attributes. ``` from opentelemetry import trace # Assuming you have a tracer and a span tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("example-span") as span: # Set your custom metric as an attribute span.set_attribute("custom_metric", 42) ``` 2. *Ensure Span Export*: Make sure your span is being exported to Phoenix. This typically involves configuring a span processor and exporter, such as the `OTLPSpanExporter`, to send spans to the Phoenix server. By setting the metric as an attribute, it will be included in the span data sent to Phoenix, and you can view it in the dashboard. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|Customizing Spans>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM I already have this, but is there anyway I can take this route and have the metric appear on the "feedback" column?
RunLLM
APP
·
To have your custom metric appear in the "feedback" column on the Phoenix dashboard, you need to log it as an annotation rather than just a span attribute. The feedback column is specifically designed to display annotations, which include labels, scores, and explanations. Here's how you can achieve this: 1. *Create Annotation*: Instead of using `set_attribute`, create an annotation with your metric details. An annotation can include a label, score, and explanation, which are displayed in the feedback column. 2. *Log Annotation*: Use the Phoenix API to log this annotation. This can be done in real-time by integrating the annotation logging within your application logic. Unfortunately, using `set_attribute` alone will not populate the feedback column, as it requires annotations. You can automate the annotation process by integrating it into your application logic, but it will still involve using the Phoenix API to log the annotation. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
how do i retrieve my current span or parent trace ID? RunLLM
RunLLM
APP
·
To retrieve the current span or parent trace ID in your application, you can use the OpenTelemetry API. Here's how you can do it: 1. *Get Current Span*: Use the `get_current_span` method from the OpenTelemetry API to retrieve the current span. 2. *Retrieve Trace ID*: Once you have the current span, you can access its trace ID from the span's context. Here's a code example: ``` from opentelemetry import trace # Get the current span current_span = trace.get_current_span() # Retrieve the trace ID from the current span trace_id = current_span.get_span_context().trace_id # Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x') print(f"Trace ID: {trace_id_hex}") ``` This code snippet retrieves the current span and extracts the trace ID, converting it to a hexadecimal string for readability. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/issues/3775|How to Get Current Span and Trace ID>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM so when I'm trying to log annotations, should I focus on logging them to the specific span or the parent trace of the span? am i understanding it correctly?
RunLLM
APP
·
When logging annotations in Phoenix, you should focus on logging them to the specific span rather than the parent trace. Annotations are designed to provide detailed feedback or metrics about specific operations represented by spans. By attaching annotations to individual spans, you can capture granular insights about each part of the trace. According to the <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Phoenix documentation>, annotations are associated with spans using the `span_id`. This allows you to target specific spans within a trace for detailed feedback or evaluation. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback>
💡 Hint: Mention RunLLM in the thread for followups.

Samanvya T.

John G. Thank you! I'm almost done with that, but one thing I notice is the Span ID I'm reporting to is not the same as the Span ID I get when I copy span ID from the dashboard.

# Get the span ID for Phoenix annotations
        span_context = span.get_span_context()
        span_id = span_context.span_id
        
        # Convert span_id to hex string format expected by Phoenix
        span_id_hex = format(span_id, '032x')

Samanvya T.
·
Okay my bad, it is correct.
🙌1
Samanvya T.
·
I'm getting extra prepended 0s: 0000000000000000f3291f2d8b724e1e vs f3291f2d8b724e1e. The bot said use '032x' but it should be '016x'
Samanvya T.
·
RunLLM how do i get the trace_id of the parent span?
RunLLM
APP
·
To get the `trace_id` of the parent span, you can access the `trace_id` from the current span's context, as the `trace_id` is shared across all spans within the same trace. Here's how you can do it: 1. *Get Current Span*: Use the `get_current_span` method to retrieve the current span. 2. *Access Trace ID*: The `trace_id` is part of the span's context and can be accessed directly. Here's a code example: ``` from opentelemetry import trace # Get the current span current_span = trace.get_current_span() # Retrieve the trace ID from the current span's context trace_id = current_span.get_span_context().trace_id # Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x') print(f"Trace ID: {trace_id_hex}") ``` This code retrieves the trace ID from the current span's context, which is the same for all spans in the trace, including the parent span. Would you like more detail on any specific part? Sources: - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089|https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089> - <https://github.com/Arize-ai/openinference/blob/main/python/openinference-instrumentation/tests/test_helpers.py|https://github.com/Arize-ai/openinference/blob/main/python/openinference-instrumentation/tests/test_helpers.py>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
Yes John G., 016x is the one that works for me.
:thankyou:1

RunLLM
APP
·
To log your own metrics per span and have them appear in the feedback column on the Phoenix dashboard, you can use annotations. Annotations allow you to attach custom metrics to spans. Here's a concise breakdown: 1. *Create Annotations*: Use the Phoenix API to create annotations for your spans. An annotation can include a label, score, and explanation, which can represent your custom metric. 2. *Log Annotations*: Send these annotations to Phoenix using the `/v1/span_annotations` endpoint. Ensure that each annotation is associated with the correct `span_id`. For example, you can create an annotation with a custom metric like this: ``` import httpx client = httpx.Client() annotation_payload = { "data": [ { "span_id": "your_span_id", "name": "custom_metric", "annotator_kind": "HUMAN", "result": {"label": "metric_label", "score": 1, "explanation": "Metric explanation"}, "metadata": {}, } ] } headers = {'api_key': '<your phoenix api key> '} client.post( "https://app.phoenix.arize.com/v1/span_annotations?sync=false", json=annotation_payload, headers=headers ) ``` This will log your custom metric as an annotation, which will then appear in the feedback column on the Phoenix dashboard. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback> - <https://docs.arize.com/phoenix/tracing/features-tracing/how-to-annotate-traces#S%3A3|How to Annotate Traces>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM what if I want to do in real-time? I have a method where I've defined a span, it's attributes etc. That's where i get the value of the metric, and I want to log it from there like span.set_attribute(), without having to do this client.post()
RunLLM
APP
·
To log metrics in real-time directly within a span, you can use the `set_attribute` method on the span to attach your custom metric as an attribute. This approach allows you to add the metric value directly to the span without making an additional HTTP request. Here's how you can do it: 1. *Set Attribute on Span*: Use the `set_attribute` method to add your custom metric as an attribute to the span. This will ensure that the metric is logged as part of the span's attributes. ``` from opentelemetry import trace # Assuming you have a tracer and a span tracer = trace.get_tracer(__name__) with tracer.start_as_current_span("example-span") as span: # Set your custom metric as an attribute span.set_attribute("custom_metric", 42) ``` 2. *Ensure Span Export*: Make sure your span is being exported to Phoenix. This typically involves configuring a span processor and exporter, such as the `OTLPSpanExporter`, to send spans to the Phoenix server. By setting the metric as an attribute, it will be included in the span data sent to Phoenix, and you can view it in the dashboard. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|Customizing Spans>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM I already have this, but is there anyway I can take this route and have the metric appear on the "feedback" column?
RunLLM
APP
·
To have your custom metric appear in the "feedback" column on the Phoenix dashboard, you need to log it as an annotation rather than just a span attribute. The feedback column is specifically designed to display annotations, which include labels, scores, and explanations. Here's how you can achieve this: 1. *Create Annotation*: Instead of using `set_attribute`, create an annotation with your metric details. An annotation can include a label, score, and explanation, which are displayed in the feedback column. 2. *Log Annotation*: Use the Phoenix API to log this annotation. This can be done in real-time by integrating the annotation logging within your application logic. Unfortunately, using `set_attribute` alone will not populate the feedback column, as it requires annotations. You can automate the annotation process by integrating it into your application logic, but it will still involve using the Phoenix API to log the annotation. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
how do i retrieve my current span or parent trace ID? RunLLM
RunLLM
APP
·
To retrieve the current span or parent trace ID in your application, you can use the OpenTelemetry API. Here's how you can do it: 1. *Get Current Span*: Use the `get_current_span` method from the OpenTelemetry API to retrieve the current span. 2. *Retrieve Trace ID*: Once you have the current span, you can access its trace ID from the span's context. Here's a code example: ``` from opentelemetry import trace # Get the current span current_span = trace.get_current_span() # Retrieve the trace ID from the current span trace_id = current_span.get_span_context().trace_id # Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x') print(f"Trace ID: {trace_id_hex}") ``` This code snippet retrieves the current span and extracts the trace ID, converting it to a hexadecimal string for readability. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/issues/3775|How to Get Current Span and Trace ID>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
RunLLM so when I'm trying to log annotations, should I focus on logging them to the specific span or the parent trace of the span? am i understanding it correctly?
RunLLM
APP
·
When logging annotations in Phoenix, you should focus on logging them to the specific span rather than the parent trace. Annotations are designed to provide detailed feedback or metrics about specific operations represented by spans. By attaching annotations to individual spans, you can capture granular insights about each part of the trace. According to the <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Phoenix documentation>, annotations are associated with spans using the `span_id`. This allows you to target specific spans within a trace for detailed feedback or evaluation. Would you like more detail on any specific part? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-interact-with-traces/capture-feedback.md|Capture Feedback>
💡 Hint: Mention RunLLM in the thread for followups.

Samanvya T.

John G. Thank you! I'm almost done with that, but one thing I notice is the Span ID I'm reporting to is not the same as the Span ID I get when I copy span ID from the dashboard.

# Get the span ID for Phoenix annotations
        span_context = span.get_span_context()
        span_id = span_context.span_id
        
        # Convert span_id to hex string format expected by Phoenix
        span_id_hex = format(span_id, '032x')

Samanvya T.
·
Okay my bad, it is correct.
🙌1
Samanvya T.
·
I'm getting extra prepended 0s: 0000000000000000f3291f2d8b724e1e vs f3291f2d8b724e1e. The bot said use '032x' but it should be '016x'
Samanvya T.
·
RunLLM how do i get the trace_id of the parent span?
RunLLM
APP
·
To get the `trace_id` of the parent span, you can access the `trace_id` from the current span's context, as the `trace_id` is shared across all spans within the same trace. Here's how you can do it: 1. *Get Current Span*: Use the `get_current_span` method to retrieve the current span. 2. *Access Trace ID*: The `trace_id` is part of the span's context and can be accessed directly. Here's a code example: ``` from opentelemetry import trace # Get the current span current_span = trace.get_current_span() # Retrieve the trace ID from the current span's context trace_id = current_span.get_span_context().trace_id # Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x') print(f"Trace ID: {trace_id_hex}") ``` This code retrieves the trace ID from the current span's context, which is the same for all spans in the trace, including the parent span. Would you like more detail on any specific part? Sources: - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089|https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089> - <https://github.com/Arize-ai/openinference/blob/main/python/openinference-instrumentation/tests/test_helpers.py|https://github.com/Arize-ai/openinference/blob/main/python/openinference-instrumentation/tests/test_helpers.py>
💡 Hint: Mention RunLLM in the thread for followups.
Samanvya T.
·
Yes John G., 016x is the one that works for me.
:thankyou:1