I have a doubt regarding logging our own metrics per span. I see annotations is a way, but it's not clear on how I can log my own metric within a span (for it to appear on the dashboard in the feedback column.
To log your own metrics per span and have them appear in the feedback column on the Phoenix dashboard, you can use annotations. Annotations allow you to attach custom metrics to spans. Here's a concise breakdown: 1. Create Annotations: Use the Phoenix API to create annotations for your spans. An annotation can include a label, score, and explanation, which can represent your custom metric.2. Log Annotations: Send these annotations to Phoenix using the /v1/span_annotations endpoint. Ensure that each annotation is associated with the correct span_id.For example, you can create an annotation with a custom metric like this: LBKSimport httpxLBKSclient = httpx.Client()LBKSannotation_payload = { "data": [ { "span_id": "your_span_id", "name": "custom_metric", "annotator_kind": "HUMAN", "result": {"label": "metric_label", "score": 1, "explanation": "Metric explanation"}, "metadata": {}, } ] }LBKSheaders = {'api_key': '<your phoenix api key> '}LBKSclient.post( "https://app.phoenix.arize.com/v1/span_annotations?sync=false", json=annotation_payload, headers=headers )LBKSThis will log your custom metric as an annotation, which will then appear in the feedback column on the Phoenix dashboard.Would you like more detail on any specific part?Sources:
- Capture Feedback
- How to Annotate Traces
馃挕聽Hint: Mention RunLLM in the thread for followups.
RunLLM what if I want to do in real-time? I have a method where I've defined a span, it's attributes etc. That's where i get the value of the metric, and I want to log it from there like span.set_attribute(), without having to do this client.post()
To log metrics in real-time directly within a span, you can use the set_attribute method on the span to attach your custom metric as an attribute. This approach allows you to add the metric value directly to the span without making an additional HTTP request. Here's how you can do it: 1. Set Attribute on Span: Use the set_attribute method to add your custom metric as an attribute to the span. This will ensure that the metric is logged as part of the span's attributes.LBKSfrom opentelemetry import traceLBKS# Assuming you have a tracer and a span tracer = trace.get_tracer(__name__)LBKSwith tracer.start_as_current_span("example-span") as span: # Set your custom metric as an attribute span.set_attribute("custom_metric", 42)LBKS2. Ensure Span Export: Make sure your span is being exported to Phoenix. This typically involves configuring a span processor and exporter, such as the OTLPSpanExporter, to send spans to the Phoenix server.By setting the metric as an attribute, it will be included in the span data sent to Phoenix, and you can view it in the dashboard.Would you like more detail on any specific part?Sources:
- Customizing Spans
馃挕聽Hint: Mention RunLLM in the thread for followups.
RunLLM I already have this, but is there anyway I can take this route and have the metric appear on the "feedback" column?
To have your custom metric appear in the "feedback" column on the Phoenix dashboard, you need to log it as an annotation rather than just a span attribute. The feedback column is specifically designed to display annotations, which include labels, scores, and explanations. Here's how you can achieve this: 1. Create Annotation: Instead of using set_attribute, create an annotation with your metric details. An annotation can include a label, score, and explanation, which are displayed in the feedback column.2. Log Annotation: Use the Phoenix API to log this annotation. This can be done in real-time by integrating the annotation logging within your application logic.Unfortunately, using set_attribute alone will not populate the feedback column, as it requires annotations. You can automate the annotation process by integrating it into your application logic, but it will still involve using the Phoenix API to log the annotation.Would you like more detail on any specific part?Sources:
- Capture Feedback
馃挕聽Hint: Mention RunLLM in the thread for followups.
how do i retrieve my current span or parent trace ID? RunLLM
To retrieve the current span or parent trace ID in your application, you can use the OpenTelemetry API. Here's how you can do it: 1. Get Current Span: Use the get_current_span method from the OpenTelemetry API to retrieve the current span.2. Retrieve Trace ID: Once you have the current span, you can access its trace ID from the span's context.Here's a code example: LBKSfrom opentelemetry import traceLBKS# Get the current span current_span = trace.get_current_span()LBKS# Retrieve the trace ID from the current span trace_id = current_span.get_span_context().trace_idLBKS# Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x')LBKSprint(f"Trace ID: {trace_id_hex}")LBKSThis code snippet retrieves the current span and extracts the trace ID, converting it to a hexadecimal string for readability.Would you like more detail on any specific part?Sources:
- How to Get Current Span and Trace ID
馃挕聽Hint: Mention RunLLM in the thread for followups.
RunLLM so when I'm trying to log annotations, should I focus on logging them to the specific span or the parent trace of the span? am i understanding it correctly?
When logging annotations in Phoenix, you should focus on logging them to the specific span rather than the parent trace. Annotations are designed to provide detailed feedback or metrics about specific operations represented by spans. By attaching annotations to individual spans, you can capture granular insights about each part of the trace.According to the Phoenix documentation, annotations are associated with spans using the span_id. This allows you to target specific spans within a trace for detailed feedback or evaluation.Would you like more detail on any specific part?Sources:
- Capture Feedback
馃挕聽Hint: Mention RunLLM in the thread for followups.
John G. Thank you! I'm almost done with that, but one thing I notice is the Span ID I'm reporting to is not the same as the Span ID I get when I copy span ID from the dashboard.
# Get the span ID for Phoenix annotations
span_context = span.get_span_context()
span_id = span_context.span_id
# Convert span_id to hex string format expected by Phoenix
span_id_hex = format(span_id, '032x')I'm getting extra prepended 0s: 0000000000000000f3291f2d8b724e1e vs f3291f2d8b724e1e. The bot said use '032x' but it should be '016x'
RunLLM how do i get the trace_id of the parent span?
To get the trace_id of the parent span, you can access the trace_id from the current span's context, as the trace_id is shared across all spans within the same trace. Here's how you can do it: 1. Get Current Span: Use the get_current_span method to retrieve the current span.
2. Access Trace ID: The trace_id is part of the span's context and can be accessed directly.Here's a code example: LBKSfrom opentelemetry import traceLBKS# Get the current span current_span = trace.get_current_span()LBKS# Retrieve the trace ID from the current span's context trace_id = current_span.get_span_context().trace_idLBKS# Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x')LBKSprint(f"Trace ID: {trace_id_hex}")LBKSThis code retrieves the trace ID from the current span's context, which is the same for all spans in the trace, including the parent span.Would you like more detail on any specific part?Sources:
- https://arize-ai.slack.com/archives/C04R3GXC8HK/p1724796536.922089
- https://github.com/Arize-ai/openinference/blob/main/python/openinference-instrumentation/tests/test_helpers.py
馃挕聽Hint: Mention RunLLM in the thread for followups.