Best Practices for Tracing Chat Conversations in Chains

Rémi C. · 2024-04-11T04:28:09.775Z

Hey what do you recommand for tracing chat conversations? Should one chain hold the full conversation. Or should one chain hold just one interaction?

Best Practices for Tracing Chat Conversations in Chains | Arize AI Community

Mikyo
·
Good question Remi, we’re actually going to support sessions and conversations in the coming months. We are thinking you correlate traces with a session id. This lets you view conversations as individual interactions as well as a top level thread. Does this align with what you were thinking?
Rémi C.
·
Meaning I just need to add a session ID to the chain, and then it reconstructs the history of a conversation (even if the full conversaion is not happening in the same chain span context) ? That would be nice yes :)
1
Rémi C.
·
But what do you recommand for now? One chain per interaction?
Mikyo
·
I think so yeah. You can add a conversation ID or session ID to the metadata attributes. You should then be able to recreate the conversation via filters
Rémi C.
·
makes sense thank you 🙂
1
Rémi C.
·
Another question I'm wondering: I want to use chainlit to implement a chat frontend, is there a way I can save the dataframe after each message?
Rémi C.
·
(I also want to hook up the feedback functions to the eval function of phoenix)
Mikyo
·
We have persistence coming pretty soon but no there’s no easy way to hook in to the completion of a trace. You definitely can save on a periodic basis.
Mikyo
·
Yeah the human feedback comes up for us for sure. We will have a way of annotating traces and conversations with human feedback but right now it’s a bit difficult
Rémi C.
·
Erf I understand persistence is coming, but I'm shipping something before tuesday haha so I'm gonna hack something
👍1
Rémi C.
·
So there's no way to know when I'm sending the trace what the span id is, is that correct?
Mikyo
·
It’s not entirely impossible I think - I think it would probably be easier though if you controlled your own ID system so you could join back the feedback on an ID you control (add the id to the metadata then join back the feedback and log as an eval). It I was on a time crunch I’d try that. Sorry for now having an elegant solution for ya
Rémi C.
·
Thank you 🙂 That confirms this is the way to go
🙏1
Rémi C.
·
Assuming I'm running on the same node, can I access px.Client() from the code outside of the notebook?
Rémi C.
·
Or does the client need to come from the notebook that launched phoenix?

17 comments