Integrating Code and Annotate Options in Phoenix Traces

Vinuka · 2024-08-14T03:41:04.758Z

Hi, Using manual instrumentation I created phoenix traces for by chatbot application. Here are the screenshots of it. But in the official Phoenix website (phoenix.arize.com (http://phoenix.arize.com)), in 'See in Action', in the traces it shows a 'Code' option to get the span and the trace id and an 'Annotate' option. How can I add it to my traces to get those options ? I have included the screenshots of my traces and the traces shown in the site. Can someone help me with that please. Thanks

RunLLM
APP
·
To add the 'Code' and 'Annotate' options to your traces in Phoenix, you need to ensure that your spans are properly instrumented and include the necessary metadata. Here are the steps to achieve this:
1. Ensure Proper Instrumentation: Make sure your spans are created with the necessary metadata. You can use the OpenTelemetry API to add custom attributes to your spans.
2. Add Metadata for 'Code' and 'Annotate' Options: You need to include specific metadata in your spans to enable these options. This involves adding attributes that Phoenix can recognize and use to display these options.
3. Update Your Tracer Configuration: Ensure your tracer is configured to include these attributes when creating spans.
Here is an example of how you can manually instrument your spans to include the necessary metadata:
```
from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import SimpleSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
tracer_provider = TracerProvider() trace.set_tracer_provider(tracer_provider)
span_exporter = OTLPSpanExporter(endpoint="[http://localhost::6006/v1/traces](http:://localhost:6006/v1/traces)") span_processor = SimpleSpanProcessor(span_exporter) tracer_provider.add_span_processor(span_processor)
tracer = trace.get_tracer(name)
with tracer.start_as_current_span("example-span") as span: span.set_attribute("code", "example_code") span.set_attribute("annotate", "example_annotation") # Add other necessary attributes span.set_attribute("example_key", "example_value")
LBKS4. **Log Evaluations**: If you want to include evaluations in the feedback column, you can log evaluations using the `log_evaluations` method. LBKS``` from phoenix.trace import SpanEvaluations import phoenix as px client = px.Client() spans_df = client.get_spans_dataframe() # Assuming 'output' column exists and you want to update it with the last LLM call output spans_df['output'] = spans_df.apply(lambda row: get_last_llm_output(row), axis=1) # Combine evaluations def combine_evaluations(trace_id): evaluations = get_evaluations_for_trace(trace_id) return combine(evaluations) spans_df['feedback'] = spans_df['trace_id'].apply(combine_evaluations) # Log updated evaluations client.log_evaluations(SpanEvaluations(eval_name="Custom Feedback", dataframe=spans_df))
Would you like more detail on any specific part?
Sources: - GitHub Issue #2705 - GitHub Issue #2696
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
Thank you so much.
Mikyo
·
Hi Vinuka, that’s just the latest version of Phoenix. You will get it if you upgrade the version
💡1
Vinuka
·
Mikyo ok sure
Vinuka
·
Mikyo arize-phoenix = {extras = ["evals"], version = "^4.6.2"} This is the phoenix version that I'm using with my application. That is the compatible version with the python and Django libraries I am using. So is there a way to get the Trace ID programmatically for a trace ?
Mikyo
·
RunLLM how do you programmatically get the trace ID for OpenTelemetry using python?
RunLLM
APP
·
To programmatically get the trace ID for a trace in OpenTelemetry using Python, you can use the get_current_span method from the OpenTelemetry API. Here is an example:
1. Install OpenTelemetry: Ensure you have the necessary OpenTelemetry packages installed.
```
pip install opentelemetry-api opentelemetry-sdk
LBKS2. **Get the Current Span and Trace ID**: Use the `get_current_span` method to retrieve the current span and its trace ID. LBKS``` from opentelemetry import trace # Get the current span current_span = trace.get_current_span() # Get the trace ID from the current span trace_id = current_span.get_span_context().trace_id # Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x') print(f"Trace ID: {trace_id_hex}")
This code snippet retrieves the current span and extracts the trace ID, converting it to a hexadecimal string for readability.
Would you like more detail on any specific part?
Sources: - GitHub Issue #3775
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
Ok Thanks
Vinuka
·
RunLLM is there a way to get the project id as well for each project to get the full trace link. This is the format of my trace link. To get it with the response, how can I get the project id as well ?
http://localhost:{get_env_port()}/projects/{projectid}/traces/{traceId}
RunLLM
APP
·
To programmatically get the project ID for each project in Phoenix, you can use a GraphQL query. Here is an example of how to do this:
1. Install Required Packages: Ensure you have the httpx package installed.
```
pip install httpx
LBKS2. **GraphQL Query to Get Project ID**: Use the following code to query the project ID based on the project name. LBKS``` from typing import Optional from httpx import Client client = Client(base_url="[http://localhost:6006](http://localhost:6006)") query = """ query ($after: String = null) { projects(after: $after) { edges { project: node { id name } } pageInfo { hasNextPage endCursor } } } """ def get_id_for_project_name(name: str) -> Optional[str]: has_next_page = True after = None while has_next_page: response = [client.post](http://client.post)( "/graphql", json={"query": query, "variables": {"after": after}} ) if (status_code := response.status_code) != 200: raise Exception(f"Failed with status code: {status_code}") data = response.json()["data"] edges = data["projects"]["edges"] projects = map(lambda edge: edge["project"], edges) for project in projects: if project["name"] == name: return project["id"] page_info = data["projects"]["pageInfo"] has_next_page = page_info["hasNextPage"] after = page_info["endCursor"] return None project_name = "your_project_name" project_id = get_id_for_project_name(project_name) print(f"Project ID: {project_id}")
This code will retrieve the project ID for a given project name. You can then use this project ID along with the trace ID to construct the full trace link.
Would you like more detail on any specific part?
Sources: - Slack Conversation
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
RunLLM how to get the project name ?
RunLLM
APP
·
To get the project name programmatically, you can use a similar GraphQL query to list all projects and their names. Here is an example of how to do this:
1. GraphQL Query to Get Project Names: Use the following code to query the project names.
```
from httpx import Client
client = Client(base_url="[http://localhost::6006](http:://localhost:6006)")
query = """ query { projects { edges { project: node { id name } } } } """
def get_project_names(): response = client.post("/graphql", json={"query": query}) if response.status_code != 200: raise Exception(f"Failed with status code: {response.status_code}") data = response.json()["data"] edges = data["projects"]["edges"] project_names = [edge["project"]["name"] for edge in edges] return project_names
project_names = get_project_names() print(f"Project Names: {project_names}")
LBKSThis code will retrieve and print the names of all projects. You can then use these names to get the corresponding project IDs as needed. Would you like more detail on any specific part? Sources: - [GitHub Issue #2379](https://github.com/Arize-ai/phoenix/issues/2379)
💡 Hint: Mention RunLLM in the thread for followups.

Vinuka

RunLLM in the attached image you can see there are several projects. So how to get the project that i am currently working on to get the trace link correctly.

# Get project names and select a project name
project_names = get_project_names()
if not project_names:
    raise Exception("No project names found")
# Choose the first project name or apply your logic to select one
project_name = project_names[2]

# Get project ID dynamically
project_id = get_id_for_project_name(project_name)
if project_id is None:
    raise Exception(f"Project ID for '{project_name}' not found")

Here I am getting the 3rd project. But if the user is working in another project, how to get that exact project that the user is working on ?

RunLLM
APP
·
To dynamically determine the project that the user is currently working on, you can use environment variables or session-specific data to identify the active project. Here’s how you can achieve this:
1. Set Environment Variable for Current Project: Ensure that the current project name is set as an environment variable when the user starts working on a project.
```
import os
os.environ["CURRENT_PROJECT_NAME"] = "your_current_project_name"
LBKS2. **Retrieve the Current Project Name**: Use the environment variable to get the current project name programmatically. LBKS``` def get_current_project_name() -> str: project_name = os.getenv("CURRENT_PROJECT_NAME") if not project_name: raise Exception("Current project name is not set") return project_name # Get the current project name current_project_name = get_current_project_name()
3. Get Project ID for the Current Project: Use the previously defined function to get the project ID for the current project name.
```
current_project_id = get_id_for_project_name(current_project_name) if current_project_id is None: raise Exception(f"Project ID for '{current_project_name}' not found")
print(f"Current Project ID: {current_project_id}")
LBKSHere is the complete code snippet:
RunLLM
APP
·
import os from typing import Optional from httpx import Client client = Client(base_url="[http://localhost:6006](http://localhost:6006)") query = """ query ($after: String = null) { projects(after: $after) { edges { project: node { id name } } pageInfo { hasNextPage endCursor } } } """ def get_id_for_project_name(name: str) -> Optional[str]: has_next_page = True after = None while has_next_page: response = [client.post](http://client.post)( "/graphql", json={"query": query, "variables": {"after": after}} ) if (status_code := response.status_code) != 200: raise Exception(f"Failed with status code: {status_code}") data = response.json()["data"] edges = data["projects"]["edges"] projects = map(lambda edge: edge["project"], edges) for project in projects: if project["name"] == name: return project["id"] page_info = data["projects"]["pageInfo"] has_next_page = page_info["hasNextPage"] after = page_info["endCursor"] return None def get_current_project_name() -> str: project_name = os.getenv("CURRENT_PROJECT_NAME") if not project_name: raise Exception("Current project name is not set") return project_name # Set the current project name as an environment variable os.environ["CURRENT_PROJECT_NAME"] = "your_current_project_name" # Get the current project name current_project_name = get_current_project_name() # Get project ID dynamically for the current project current_project_id = get_id_for_project_name(current_project_name) if current_project_id is None: raise Exception(f"Project ID for '{current_project_name}' not found") print(f"Current Project ID: {current_project_id}")
This approach ensures that you always get the project ID for the project that the user is currently working on.
Would you like more detail on any specific part?
Sources: - Slack Conversation
💡 Hint: Mention RunLLM in the thread for followups.

Integrating Code and Annotate Options in Phoenix Traces

17 comments

Integrating Code and Annotate Options in Phoenix Traces

17 comments