Exploring Arize for Tracing RAG System at Adviso, Montreal

·May 21, 2025 08:09 PM

Hello everyone, I am Mickael, currently exploring arize for our tracing RAG system in Adviso, Montreal marketing company 🙂

5 comments

· Sorted by Oldest

John G.
·
Hey Michael, welcome! If you haven't found it already, we have an example RAG tracing and eval workflow in Arize here: https://docs.arize.com/arize/cookbooks/code-examples/applications/rag
Mickael W.
·
thank you ! I was wondering if there was a ressource specific to langgraph. because in langgraph we have consequent states and I am wondering if I can select what can be considered by arize at each langgraph node. 🙂 the langgraph format is really not convenient with the Arize evaluation system ^^
John G.
·
Hmm I don't have a specific RAG eval example with LangGraph, though we have a set of some examples of LangGraph evals here that might help They do have their own tracing paradigm that we try our best to combine with, so it does create some rough edges. We can put together an example of rag evals. Is there a specific vector db you're using or any other confounding factors that would be helpful to see? Feel free to share via DM if you'd rather
Mickael W.
·
No I think it is good to share if someone else have same 'issue' 🙂 I will port that convo on the dedicated thread arize-platform-support 😉 inside every nodes of the langgraph I am using langchain as much as possible very high level stack looks like this :
get message history stored on Firestore
use the init_chat_model of langchain with openAI models
then use langchain_pinecone PineconeVectorStore (but it might soon change to qdrant)
complete search with web crawl using GoogleSearchAPIWrapper and Firecrawl api (without their sdk, direct curl)
Using langgraph mostly to simplify the async "parallel" calls to API 🙂
John G.
·
Awesome thank you for that! Here's the support thread for those finding this message later: https://arize-ai.slack.com/archives/C017QTPKE0H/p1747927486618999?thread_ts=1747888679.562029&cid=C017QTPKE0H We can centralize the convo there

Exploring Arize for Tracing RAG System at Adviso, Montreal | Arize AI Community