Eric C.

Hi everyone, forgive me for the looong text. 😅 I’ve been exploring different ways to integrate a test project with Phoenix OSS using OpenTelemetry to identify how my team can use Phoenix effectively in paralell with other open telemetry backend used to monitor the application. So I built a "pet project" used to test different ways to feed both backends simultaneously: Phoenix (for tracing and evals) and Jaeger (for general application and infrastructure monitoring). After some trial and error, and with help from 🔒[private user] and Anthony P., I ended with two integration approaches:

Ultimately, I couldn't get Phoenix to behave as expected in either case. I’d like to confirm if I’m missing something or if these are current practical limitations in current UI/UX. Starting with strategy #2: While manually managing separate contexts works, it is much more complex and prone to errors. I couldn't send/capture all spans properly when using frameworks like strandsagents or LiteLLM proxy, which plug directly into the global OpenTelemetry SDK. Therefore, I’m ruling this out as a viable solution. The issue with strategy #1 (Single Global Context / Dual Providers): This approach works well only if I initialize telemetry exactly when the code starts agent tasks. Phoenix seems to expect OpenInference attributes to be present on the root span (please see the attached images). However, in a "real-world" application, the root span is created with application data (e.g., an HTTP handler or Lambda entry point) before any prompt, LLM, or agent data is captured. The spans containing the OpenInference info will always be child spans, never the root. To illustrate this, I ran a "dialogue" via a pytest test (where the agent invocation produces the root span) and the same dialogue via the "application" (a Lambda function, where the agent is a child span from the application span): pytest scenarios (Agent = Root Span):

Application scenarios (Agent = Child Span):

My Questions:

Any insights would be appreciated, and big thanks in advance to all (and in particular to 🔒[private user] and Anthony P. for their help)!

Hi guys. I'm working on a chat agent using StrandsAgents deployed as AWS Lambdas that uses a self-hosted LiteLLM Proxy to interact with the Models. This environment already uses opentelemetry with a Jaeger service to monitor application issues, overall metrics, latency, etc. I'd like to use Phoenix OSS to collect the model statistics and prompts interactions so I can analyze and improve the conversation flows in Phoneix. The setup I made is pretty standard, using Phoneix environment variables and manually set two trace providers, one for the app e other for the StrandsAgents, but in phoneix I'm getting all spans making the trace harder to analyze. Is there a way to set the telemetry so te phoneix tracer provider to create it's span indenependently from the global tracer provider which is feeding the app provider? Also it seems that not all information are being properly ingested into Phoenix. Columns, "Kind", "Status", "Tokens" does not look like to being populated. below is my current setup with some code snippets:

-- python (otel.py)
import os

from typing import Dict, Optional
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

from strands.telemetry import StrandsTelemetry
from phoenix.otel import register

phoenix_tracer_provider = register()
strands_telemetry = StrandsTelemetry(tracer_provider=phoenix_tracer_provider)
strands_telemetry.setup_meter(enable_otlp_exporter=True)
strands_telemetry.setup_otlp_exporter().setup_console_exporter()

app_provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
app_provider.add_span_processor(processor)

# Sets the global default tracer provider
trace.set_tracer_provider(app_provider)
tracer = trace.get_tracer(__name__)

Than in my lambda handlers I just import the tracer and use it as usual.

--- python (agent.py)
from .otel import tracer

@tracer.start_as_current_span("lambda_handler")
def handler(event: Dict[str, Any], _context) -> Dict[str, Any]:
    span = trace.get_current_span()
    span.add_event("some application info")
    ...
    span.set_attribute("session.id", user_input.session_id)
    span.set_attribute("user.message", message_str)
    ...
    session_params = {
      "session_id": user_input.session_id,
      "bucket": settings.agents.session_bucket,
      "prefix": f"{settings.agents.session_prefix}/agente-triagem",
    }

    agent = Agent(
      system_prompt=system_prompt,
      model=model,
      session_manager=session_manager,
      conversation_manager=conversation_manager,
      structured_output_model=TriagemOutput,
      tools=[],
      trace_attributes=trace_attributes,
    )

    result = agent(message_str)

Hi all, not really an Arize AI question, but I believe this community would have valuable insights in this matter. I was thinking about caching in LLM applications and its implications for user data privacy/security. Suppose I have a chat server application that serves multiple users simultaneously with prompt caching at the proxy level (e.g., using LiteLLM) or even without proxy but using a single API key for the entire application (which would use the provider API internal caching), is there a risk that user-specific data could be inadvertently cached and then served to other users? Example: user A's query invoke a tool calling to retreive his account balance and then the LLM generated response is cached. User B makes a similar query that hits the cache? Could the response containing user A's account balance be served to user B? If yes, is there a mechanism to prevent this from happening? Has anyone faced this kind of issue in production systems? Any insights on how developers are controling the LLM caching across different user sessions or even between different phases (ie application cache versus user cache)?

hi all! I'm Eric from Brazil. I'm taking my first steps using phoenix. I have a long time developing all sort of systems and now I'm entering the world of building "AI powered applications" (dont ask em to define what this means...😆 ) I heard about phoenix couple of months ago during an eval course by Hamel & Shreya but only now I had the opportunity to start building something aimed for production. As expected from a newbie, I have ton of questions and I'm count on the experienced people here to help me out. Happy xmas to all!