Instrumenting OpenAI Calls with OpenTelemetry: Key Questions Answered | Arize AI Community

Arize AI Community Icon

Instrumenting OpenAI Calls with OpenTelemetry: Key Questions Answered | Arize AI Community

RunLLM
APP
·
1. Instrument Only Required OpenAI Calls: Use suppress_tracing to selectively disable tracing for specific OpenAI API calls. Wrap the calls you don't want to trace within the suppress_tracing context manager.2. Push Evaluation Using the Same Snippet: To push evaluation, you can add custom attributes to the spans or create new spans for evaluation metrics.Would you like more detail on either of these points?Sources: - Slack Conversation - GitHub Issue #2340
💡 Hint: Mention RunLLM in the thread for followups.
Anuraag T.
·
RunLLM how do I trace my custom retriever span using the same api?
RunLLM
APP
·
To trace your custom retriever span using the same API, you can manually create and manage spans within your retriever's logic. Here's an example of how to do this: 1. Setup Tracer: Ensure you have the tracer setup as shown in your initial code snippet. 2. Create Custom Span: Use the start_as_current_span method to create a span for your custom retriever.Here's an example: ```import openai from opentelemetry import trace from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk import trace as trace_sdk from opentelemetry.sdk.trace.export import SimpleSpanProcessor from openinference.instrumentation.openai import OpenAIInstrumentorendpoint = "[https://127.0.0.1::6006/v1/traces](https:://127.0.0.1:6006/v1/traces)"tracer_provider = trace_sdk.TracerProvider() tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint, certificate_file=r"path/to/cacert.pem")))OpenAIInstrumentor().instrument(tracer_provider=tracer_provider) tracer = trace.get_tracer(name)def custom_retriever(query): with tracer.start_as_current_span("custom_retriever_span") as span: span.set_attribute("query", query) # Your custom retriever logic here result = "retrieved data" # Replace with actual retrieval logic span.set_attribute("result", result) return resultif name == "main": client = openai.OpenAI() response = client.chat.completions.create( model="gpt-4-32k", messages=[{"role": "user", "content": "Write a haiku."}], max_tokens=20, ) print(response.choices[0].message.content)
# Call custom retriever query = "example query" retrieved_data = custom_retriever(query) print(retrieved_data)LBKS```LBKSLBKSThis code sets up a custom span for the retriever and adds attributes to it.LBKSWould you like more detail on any specific part of this process?LBKSSources:
- Manual Instrumentation Guide - Slack Discussion
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
hi Anuraag T.! It sounds like you want to use manual instrumentation with possibly sending an annotation/feedback within your application. We have a couple examples that show you how to do this:
1.
here's a full stack example https://github.com/Arize-ai/phoenix/tree/main/examples/manually-instrumented-chatbot
2.
and here's an example within a notebook https://github.com/Arize-ai/phoenix/blob/main/tutorials/human_feedback/chatbot_with_human_feedback.ipynb
In both of these examples we go over how to wire up your application to send feedback (which can be an evaluation as well) to phoenix directly alongside a manually instrumented span
Anuraag T.
·
Hi, Dustin N. Is there any documentation for v1 apis?
Dustin N.
·
yeah! if you go to <server_endpoint>/docs you should get the OpenAPI spec for our rest API
Anuraag T.
·
is it possible to pass ssl certificate file in phoenix.Client?
Dustin N.
·
we haven't wired it up explicitly, but the phoenix Client object is backed by an httpx.Client object under the _client attribute, if you reach in and modify that to include a cert with every request I think that should do what you're asking for
✅1
Dustin N.
·
https://www.python-httpx.org/advanced/ssl/#client-side-certificates