Hello everyone, I would like your help with real-time evaluation implementations. I have already managed to send my application traces to Phoenix, but I am having difficulty creating a job to run every x amount of time and perform the span evaluation.
Hello @jason, I'm using Phoenix, but when I try to run the evaluation it calls the `get_qa_with_reference` method which always returns empty since I don't have a span with this span_kid.
Yes, I am using the OpenInference specification to send my application traces. I needed to make a custom implementation, since I am using Spring AI
Hello, I have the following trace structure, in some cases including the tools span.
The attributes for the request span are.
{
"input": {
"value": "Hellow"
},
"output": {
"value": "Hello, how can I help you?"
},
"session": {
"id": "6c7916d6-ccbf-440a-a805-8a1bb3c4dc76"
},
"openinference": {
"span": {
"kind": "CHAIN"
}
}
}The other two spans follow the same attribute structure, as they are LLM.
{
"llm": {
"model_name": "gpt-4o-mini-2024-07-18",
"token_count": {
"total": 1630,
"prompt": 1610,
"completion": 20,
"prompt_details": {
"audio": 0,
"cache_read": 1408
},
"completion_details": {
"audio": 0,
"reasoning": 0
}
},
"input_messages": [
{
"message": {
"role": "system",
"content": "You are AgentMatcher..."
}
},
{
"message": {
"role": "user",
"content": "Hello"
}
}
],
"output_messages": [
{
"message": {
"role": "assistant",
"content": "{\n \"confidence\": 0.95,\n \"selectedAgent\": \"agent\"\n}"
}
}
]
},
"sessionId": "6c7916d6-ccbf-440a-a805-8a1bb3c4dc76",
"openinference": {
"span": {
"kind": "LLM"
}
}
}