Eric C.

Hi guys. I'm working on a chat agent using StrandsAgents deployed as AWS Lambdas that uses a self-hosted LiteLLM Proxy to interact with the Models. This environment already uses opentelemetry with a Jaeger service to monitor application issues, overall metrics, latency, etc. I'd like to use Phoenix OSS to collect the model statistics and prompts interactions so I can analyze and improve the conversation flows in Phoneix. The setup I made is pretty standard, using Phoneix environment variables and manually set two trace providers, one for the app e other for the StrandsAgents, but in phoneix I'm getting all spans making the trace harder to analyze. Is there a way to set the telemetry so te phoneix tracer provider to create it's span indenependently from the global tracer provider which is feeding the app provider? Also it seems that not all information are being properly ingested into Phoenix. Columns, "Kind", "Status", "Tokens" does not look like to being populated. below is my current setup with some code snippets:

-- python (otel.py)
import os

from typing import Dict, Optional
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

from strands.telemetry import StrandsTelemetry
from phoenix.otel import register

phoenix_tracer_provider = register()
strands_telemetry = StrandsTelemetry(tracer_provider=phoenix_tracer_provider)
strands_telemetry.setup_meter(enable_otlp_exporter=True)
strands_telemetry.setup_otlp_exporter().setup_console_exporter()

app_provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
app_provider.add_span_processor(processor)

# Sets the global default tracer provider
trace.set_tracer_provider(app_provider)
tracer = trace.get_tracer(__name__)

Than in my lambda handlers I just import the tracer and use it as usual.

--- python (agent.py)
from .otel import tracer

@tracer.start_as_current_span("lambda_handler")
def handler(event: Dict[str, Any], _context) -> Dict[str, Any]:
    span = trace.get_current_span()
    span.add_event("some application info")
    ...
    span.set_attribute("session.id", user_input.session_id)
    span.set_attribute("user.message", message_str)
    ...
    session_params = {
      "session_id": user_input.session_id,
      "bucket": settings.agents.session_bucket,
      "prefix": f"{settings.agents.session_prefix}/agente-triagem",
    }

    agent = Agent(
      system_prompt=system_prompt,
      model=model,
      session_manager=session_manager,
      conversation_manager=conversation_manager,
      structured_output_model=TriagemOutput,
      tools=[],
      trace_attributes=trace_attributes,
    )

    result = agent(message_str)

Hi all, not really an Arize AI question, but I believe this community would have valuable insights in this matter. I was thinking about caching in LLM applications and its implications for user data privacy/security. Suppose I have a chat server application that serves multiple users simultaneously with prompt caching at the proxy level (e.g., using LiteLLM) or even without proxy but using a single API key for the entire application (which would use the provider API internal caching), is there a risk that user-specific data could be inadvertently cached and then served to other users? Example: user A's query invoke a tool calling to retreive his account balance and then the LLM generated response is cached. User B makes a similar query that hits the cache? Could the response containing user A's account balance be served to user B? If yes, is there a mechanism to prevent this from happening? Has anyone faced this kind of issue in production systems? Any insights on how developers are controling the LLM caching across different user sessions or even between different phases (ie application cache versus user cache)?

hi all! I'm Eric from Brazil. I'm taking my first steps using phoenix. I have a long time developing all sort of systems and now I'm entering the world of building "AI powered applications" (dont ask em to define what this means...😆 ) I heard about phoenix couple of months ago during an eval course by Hamel & Shreya but only now I had the opportunity to start building something aimed for production. As expected from a newbie, I have ton of questions and I'm count on the experienced people here to help me out. Happy xmas to all!