Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Adding Custom Metrics to Phoenix Span ID for User Feedback | Arize AI Community

Phoenix Support

Adding Custom Metrics to Phoenix Span ID for User Feedback

·Apr 24, 2024 07:24 AM

Hello, Is there any way I can get the Phoenix to return span id to my application? I want to add user feedback and some other custom eval metrics to the span id.

9 comments

· Sorted by Oldest

Vipul B.
·
I see an open issue #2513. Is there any workaround I can use until this is completed?
Mikyo
·
Hi Vipul, I'd say the only workaround for human feedback right now is logging evaluations. In terms of what ID to use to send the evaluation, I think the most advisable solution would be to create your own ID and to place it in metadata attributes. This way you don't have to rely on trace / span ids from OTEL. You can then pull down spans and perform a join to send the feedback in bulk. Definitely leave us your use-case in the above issue!
Vipul B.
·
Hi Mikyo, Adding uuid in the metadata fulfils my use case. I am building a chatbot and using phoenix for observability and eval. Capturing user feedback is important part of the project. Thank you for the help!
Vipul B.
·
I was able to add metadata to LCEL during invocation using below:
chain.invoke("Hello, How are you doing?", config={'metadata': {'uuid': 1}})
The phoenix docs mention adding metadata to the LLM chain. LCEL returns a RunnableSequence which, iiuc, does not have any option to add metadata at the chain level.
Mikyo
·
Oh interesting. So I’m guessing you can’t add metadata to RunnableSequence? That’s a bit odd on LC’s part. I think querying off of invocation metadata might be a tad tough but let us know how it goes. We might have solutions for you in the near future
Vipul B.
·
querying off of invocation metadata might be a tad tough
Wondering why it would be difficult? I am not seeing any difference in setting metadata during invocation vs setting in chain (I tried both using RetrievalQA chain). Below uuid is set via invocation and application_type is set via chain
{"uuid": 1, "application_type": "question_answering"}
I am able to query the invocation metadata successfully.
🙌1
Vipul B.
·
So I'm guessing you can't add metadata to RunnableSequence? Right
Vipul B.
·
Thank you Mikyo!!
Mikyo
·
No problem!