馃敀[private user] I think I'm almost there but given the amount of manual setup required, I'm under the impression that I may be going down a bad path. with this setup phoneix is reciving and showing traces from litellm proxy, completions but still not receiviing the strands agent traces and metrics. Those keep going to the jaeger server even with the manual setup. In your experience do you think it would be better to just use a global trace feeding both service (phoenix and app otel service)? If so, what are attributes that would need to setup to feed phoenix properly? Once again thanks for yout time and help!
Hi 馃敀[private user] thanks for the help. I'm not totally familiar with the otel protocol and with your explanations things are making more sense. 馃槂 There is one point thou that I didn't fully understand: In the lambda handler I'm calling span = trace.get_current_span() to get my "application span" (the one I use to add events, attributes and child spans regarding the application logic). Should I call span.set_attribute(SpanAttributes.OPENINFERENCE_SPAN_KIND, "agent") on this span object or should I obtain a different span object from the phoneix trace provider from otel.py?
Hi guys. I'm working on a chat agent using StrandsAgents deployed as AWS Lambdas that uses a self-hosted LiteLLM Proxy to interact with the Models. This environment already uses opentelemetry with a Jaeger service to monitor application issues, overall metrics, latency, etc. I'd like to use Phoenix OSS to collect the model statistics and prompts interactions so I can analyze and improve the conversation flows in Phoneix. The setup I made is pretty standard, using Phoneix environment variables and manually set two trace providers, one for the app e other for the StrandsAgents, but in phoneix I'm getting all spans making the trace harder to analyze. Is there a way to set the telemetry so te phoneix tracer provider to create it's span indenependently from the global tracer provider which is feeding the app provider? Also it seems that not all information are being properly ingested into Phoenix. Columns, "Kind", "Status", "Tokens" does not look like to being populated. below is my current setup with some code snippets:
-- python (otel.py)
import os
from typing import Dict, Optional
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from strands.telemetry import StrandsTelemetry
from phoenix.otel import register
phoenix_tracer_provider = register()
strands_telemetry = StrandsTelemetry(tracer_provider=phoenix_tracer_provider)
strands_telemetry.setup_meter(enable_otlp_exporter=True)
strands_telemetry.setup_otlp_exporter().setup_console_exporter()
app_provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
app_provider.add_span_processor(processor)
# Sets the global default tracer provider
trace.set_tracer_provider(app_provider)
tracer = trace.get_tracer(__name__)Than in my lambda handlers I just import the tracer and use it as usual.
--- python (agent.py)
from .otel import tracer
@tracer.start_as_current_span("lambda_handler")
def handler(event: Dict[str, Any], _context) -> Dict[str, Any]:
span = trace.get_current_span()
span.add_event("some application info")
...
span.set_attribute("session.id", user_input.session_id)
span.set_attribute("user.message", message_str)
...
session_params = {
"session_id": user_input.session_id,
"bucket": settings.agents.session_bucket,
"prefix": f"{settings.agents.session_prefix}/agente-triagem",
}
agent = Agent(
system_prompt=system_prompt,
model=model,
session_manager=session_manager,
conversation_manager=conversation_manager,
structured_output_model=TriagemOutput,
tools=[],
trace_attributes=trace_attributes,
)
result = agent(message_str)Hi all, not really an Arize AI question, but I believe this community would have valuable insights in this matter. I was thinking about caching in LLM applications and its implications for user data privacy/security. Suppose I have a chat server application that serves multiple users simultaneously with prompt caching at the proxy level (e.g., using LiteLLM) or even without proxy but using a single API key for the entire application (which would use the provider API internal caching), is there a risk that user-specific data could be inadvertently cached and then served to other users? Example: user A's query invoke a tool calling to retreive his account balance and then the LLM generated response is cached. User B makes a similar query that hits the cache? Could the response containing user A's account balance be served to user B? If yes, is there a mechanism to prevent this from happening? Has anyone faced this kind of issue in production systems? Any insights on how developers are controling the LLM caching across different user sessions or even between different phases (ie application cache versus user cache)?
hi all! I'm Eric from Brazil. I'm taking my first steps using phoenix. I have a long time developing all sort of systems and now I'm entering the world of building "AI powered applications" (dont ask em to define what this means...馃槅 ) I heard about phoenix couple of months ago during an eval course by Hamel & Shreya but only now I had the opportunity to start building something aimed for production. As expected from a newbie, I have ton of questions and I'm count on the experienced people here to help me out. Happy xmas to all!
