Integrating Code and Annotate Options in Phoenix Traces

Vinuka · 2024-08-14T03:41:04.758Z

Hi, Using manual instrumentation I created phoenix traces for by chatbot application. Here are the screenshots of it. But in the official Phoenix website (phoenix.arize.com (http://phoenix.arize.com)), in 'See in Action', in the traces it shows a 'Code' option to get the span and the trace id and an 'Annotate' option. How can I add it to my traces to get those options ? I have included the screenshots of my traces and the traces shown in the site. Can someone help me with that please. Thanks

Integrating Code and Annotate Options in Phoenix Traces | Arize AI Community

RunLLM
APP
·
To add the 'Code' and 'Annotate' options to your traces in Phoenix, you need to ensure that your spans are properly instrumented and include the necessary metadata. Here are the steps to achieve this: 1. Ensure Proper Instrumentation: Make sure your spans are created with the necessary metadata. You can use the OpenTelemetry API to add custom attributes to your spans.2. Add Metadata for 'Code' and 'Annotate' Options: You need to include specific metadata in your spans to enable these options. This involves adding attributes that Phoenix can recognize and use to display these options.3. Update Your Tracer Configuration: Ensure your tracer is configured to include these attributes when creating spans.Here is an example of how you can manually instrument your spans to include the necessary metadata: LBKSfrom opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import SimpleSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporterLBKS# Configure the tracer provider tracer_provider = TracerProvider() trace.set_tracer_provider(tracer_provider)LBKS# Configure the span exporter span_exporter = OTLPSpanExporter(endpoint="[http://localhost:6006/v1/traces](http://localhost:6006/v1/traces)") span_processor = SimpleSpanProcessor(span_exporter) tracer_provider.add_span_processor(span_processor)LBKS# Get the tracer tracer = trace.get_tracer(__name__)LBKS# Create a span with custom metadata with tracer.start_as_current_span("example-span") as span: span.set_attribute("code", "example_code") span.set_attribute("annotate", "example_annotation") # Add other necessary attributes span.set_attribute("example_key", "example_value")LBKS4. Log Evaluations: If you want to include evaluations in the feedback column, you can log evaluations using the log_evaluations method.LBKSfrom phoenix.trace import SpanEvaluations import phoenix as pxLBKSclient = px.Client() spans_df = client.get_spans_dataframe()LBKS# Assuming 'output' column exists and you want to update it with the last LLM call output spans_df['output'] = spans_df.apply(lambda row: get_last_llm_output(row), axis=1)LBKS# Combine evaluations def combine_evaluations(trace_id): evaluations = get_evaluations_for_trace(trace_id) return combine(evaluations)LBKSspans_df['feedback'] = spans_df['trace_id'].apply(combine_evaluations)LBKS# Log updated evaluations client.log_evaluations(SpanEvaluations(eval_name="Custom Feedback", dataframe=spans_df))LBKSWould you like more detail on any specific part?Sources: - GitHub Issue #2705 - GitHub Issue #2696
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
Thank you so much.
Mikyo
·
Hi Vinuka, that’s just the latest version of Phoenix. You will get it if you upgrade the version
💡1
Vinuka
·
Mikyo ok sure
Vinuka
·
Mikyo arize-phoenix = {extras = ["evals"], version = "^4.6.2"} This is the phoenix version that I'm using with my application. That is the compatible version with the python and Django libraries I am using. So is there a way to get the Trace ID programmatically for a trace ?
Mikyo
·
RunLLM how do you programmatically get the trace ID for OpenTelemetry using python?
RunLLM
APP
·
To programmatically get the trace ID for a trace in OpenTelemetry using Python, you can use the get_current_span method from the OpenTelemetry API. Here is an example: 1. Install OpenTelemetry: Ensure you have the necessary OpenTelemetry packages installed.LBKSpip install opentelemetry-api opentelemetry-sdkLBKS2. Get the Current Span and Trace ID: Use the get_current_span method to retrieve the current span and its trace ID.LBKSfrom opentelemetry import traceLBKS# Get the current span current_span = trace.get_current_span()LBKS# Get the trace ID from the current span trace_id = current_span.get_span_context().trace_idLBKS# Convert the trace ID to a hexadecimal string trace_id_hex = format(trace_id, '032x')LBKSprint(f"Trace ID: {trace_id_hex}")LBKSThis code snippet retrieves the current span and extracts the trace ID, converting it to a hexadecimal string for readability.Would you like more detail on any specific part?Sources: - GitHub Issue #3775
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
Ok Thanks
Vinuka
·
RunLLM is there a way to get the project id as well for each project to get the full trace link. This is the format of my trace link. To get it with the response, how can I get the project id as well ?
http://localhost:{get_env_port()}/projects/{projectid}/traces/{traceId}
RunLLM
APP
·
To programmatically get the project ID for each project in Phoenix, you can use a GraphQL query. Here is an example of how to do this: 1. Install Required Packages: Ensure you have the httpx package installed.LBKSpip install httpxLBKS2. GraphQL Query to Get Project ID: Use the following code to query the project ID based on the project name.LBKSfrom typing import Optional from httpx import ClientLBKSclient = Client(base_url="[http://localhost:6006](http://localhost:6006)")LBKSquery = """ query ($after: String = null) { projects(after: $after) { edges { project: node { id name } } pageInfo { hasNextPage endCursor } } } """LBKSdef get_id_for_project_name(name: str) -> Optional[str]: has_next_page = True after = None while has_next_page: response = [client.post](http://client.post)( "/graphql", json={"query": query, "variables": {"after": after}} ) if (status_code := response.status_code) != 200: raise Exception(f"Failed with status code: {status_code}") data = response.json()["data"] edges = data["projects"]["edges"] projects = map(lambda edge: edge["project"], edges) for project in projects: if project["name"] == name: return project["id"] page_info = data["projects"]["pageInfo"] has_next_page = page_info["hasNextPage"] after = page_info["endCursor"] return NoneLBKSproject_name = "your_project_name" project_id = get_id_for_project_name(project_name) print(f"Project ID: {project_id}")LBKSThis code will retrieve the project ID for a given project name. You can then use this project ID along with the trace ID to construct the full trace link.Would you like more detail on any specific part?Sources: - Slack Conversation
💡 Hint: Mention RunLLM in the thread for followups.
Vinuka
·
RunLLM how to get the project name ?
RunLLM
APP
·
To get the project name programmatically, you can use a similar GraphQL query to list all projects and their names. Here is an example of how to do this: 1. GraphQL Query to Get Project Names: Use the following code to query the project names.LBKSfrom httpx import ClientLBKSclient = Client(base_url="[http://localhost:6006](http://localhost:6006)")LBKSquery = """ query { projects { edges { project: node { id name } } } } """LBKSdef get_project_names(): response = [client.post](http://client.post)("/graphql", json={"query": query}) if response.status_code != 200: raise Exception(f"Failed with status code: {response.status_code}") data = response.json()["data"] edges = data["projects"]["edges"] project_names = [edge["project"]["name"] for edge in edges] return project_namesLBKSproject_names = get_project_names() print(f"Project Names: {project_names}")LBKSThis code will retrieve and print the names of all projects. You can then use these names to get the corresponding project IDs as needed.Would you like more detail on any specific part?Sources: - GitHub Issue #2379
💡 Hint: Mention RunLLM in the thread for followups.

Vinuka

·

RunLLM in the attached image you can see there are several projects. So how to get the project that i am currently working on to get the trace link correctly.

# Get project names and select a project name
project_names = get_project_names()
if not project_names:
    raise Exception("No project names found")
# Choose the first project name or apply your logic to select one
project_name = project_names[2]

# Get project ID dynamically
project_id = get_id_for_project_name(project_name)
if project_id is None:
    raise Exception(f"Project ID for '{project_name}' not found")

Here I am getting the 3rd project. But if the user is working in another project, how to get that exact project that the user is working on ?

RunLLM
APP
·
To dynamically determine the project that the user is currently working on, you can use environment variables or session-specific data to identify the active project. Here’s how you can achieve this: 1. Set Environment Variable for Current Project: Ensure that the current project name is set as an environment variable when the user starts working on a project.LBKSimport osLBKS# Set the current project name as an environment variable os.environ["CURRENT_PROJECT_NAME"] = "your_current_project_name"LBKS2. Retrieve the Current Project Name: Use the environment variable to get the current project name programmatically.LBKSdef get_current_project_name() -> str: project_name = os.getenv("CURRENT_PROJECT_NAME") if not project_name: raise Exception("Current project name is not set") return project_nameLBKS# Get the current project name current_project_name = get_current_project_name()LBKS3. Get Project ID for the Current Project: Use the previously defined function to get the project ID for the current project name.LBKS# Get project ID dynamically for the current project current_project_id = get_id_for_project_name(current_project_name) if current_project_id is None: raise Exception(f"Project ID for '{current_project_name}' not found")LBKSprint(f"Current Project ID: {current_project_id}")LBKSHere is the complete code snippet:
RunLLM
APP
·
from typing import Optional from httpx import ClientLBKSclient = Client(base_url="[http://localhost:6006](http://localhost:6006)")LBKSquery = """ query ($after: String = null) { projects(after: $after) { edges { project: node { id name } } pageInfo { hasNextPage endCursor } } } """LBKSdef get_id_for_project_name(name: str) -> Optional[str]: has_next_page = True after = None while has_next_page: response = [client.post](http://client.post)( "/graphql", json={"query": query, "variables": {"after": after}} ) if (status_code := response.status_code) != 200: raise Exception(f"Failed with status code: {status_code}") data = response.json()["data"] edges = data["projects"]["edges"] projects = map(lambda edge: edge["project"], edges) for project in projects: if project["name"] == name: return project["id"] page_info = data["projects"]["pageInfo"] has_next_page = page_info["hasNextPage"] after = page_info["endCursor"] return NoneLBKSdef get_current_project_name() -> str: project_name = os.getenv("CURRENT_PROJECT_NAME") if not project_name: raise Exception("Current project name is not set") return project_nameLBKS# Set the current project name as an environment variable os.environ["CURRENT_PROJECT_NAME"] = "your_current_project_name"LBKS# Get the current project name current_project_name = get_current_project_name()LBKS# Get project ID dynamically for the current project current_project_id = get_id_for_project_name(current_project_name) if current_project_id is None: raise Exception(f"Project ID for '{current_project_name}' not found")LBKSprint(f"Current Project ID: {current_project_id}")LBKS```LBKSLBKSThis approach ensures that you always get the project ID for the project that the user is currently working on.LBKSWould you like more detail on any specific part?LBKSSources: - [Slack Conversation](https://arize-ai.slack.com/archives/C04R3GXC8HK/p1717704521.771589)
💡 Hint: Mention RunLLM in the thread for followups.

17 comments