Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Understanding Chat Message Type Structures for Instrumentation | Arize AI Community

Phoenix Support

Understanding Chat Message Type Structures for Instrumentation

·May 13, 2024 02:35 PM

What is the expected type structure for chat messages? The manual instrumentation example uses a generator like kind of strutctures that abstracts away the type structure of the arguments; this makes it very hard to understand what is expected? I have failed to instrument succesfully this part of my app

9 comments

· Sorted by Oldest

Xander S.
·
Hey Rémi C., they are strings that looks like "llm.input_messages.0.message.content" and "llm.input_messages.0.message.role", I believe.

·

the generator is equivalent to this

for messages_index, message in enumerate(messages):
    span.set_attribute(f"{SpanAttributes.LLM_INPUT_MESSAGES}.{messages_index}.{MessageAttributes.MESSAGE_ROLE}", message.role)
    span.set_attribute(f"{SpanAttributes.LLM_INPUT_MESSAGES}.{messages_index}.{MessageAttributes.MESSAGE_CONTENT}", message.content)

Rémi C.
·
Thank you, I was trying to pass a list of messages
🙌1
Trantor D.
·
Hello! I also had a similar question about docs earlier as well. Are the answers to these kinds of questions documented anywhere specific besides the actual code that I may have missed? Perhaps I need to do some more reading into just OpenTelemetry? etc. Want to make sure I exhaust all documentation!
Xander S.
·
Hey Trantor D., the definitive place to look is the OpenInference spec, specifically the semantic conventions.
🙌1
Trantor D.
·
Wow! Excellent! I had seen this before but I guess I forgot! This is exactly what I needed for manual instrumentation question previously - thanks!
🔥1
Rémi C.
·
In the semantic convention input_messages are stored this way
Xander S.
·
Rémi C. It think that is out of date, we will update.
🙌1
Xander S.
·
Thanks for flagging.