Troubleshooting Arize Phoenix with RAG System and Llama-Index | Arize AI Community

Arize AI Community Icon

Troubleshooting Arize Phoenix with RAG System and Llama-Index | Arize AI Community

Roger Y.
·
Hi Samantha. We’re happy to assist. In general, the set_global_handler snippet you have is the way to go. Would you mind sharing what code you are running to generate the traces?
Roger Y.
·
is there a specific function from llama-index you’re calling that you’re expecting traces from?

·

vector_store = MilvusVectorStore(...)

llm = AzureOpenAI(...)

callback_handler = OpenInferenceTraceCallbackHandler()

embed_model = HuggingFaceEmbedding()

service_context = ServiceContext.from_defaults(
    embed_model=embed_model,
    llm = llm,
    callback_manager=CallbackManager(handlers = [callback_handler])
)

index = VectorStoreIndex.from_vector_store(vector_store=vector_store,                                     service_context=service_context)

query_engine = index.as_query_engine(
                                similarity_top_k=4,
                                streaming=True)

session = px.launch_app()
rag_result = query_engine.query(prompt)

Roger Y.
·
ok thanks. i’ll take a look
Samantha F.
·
This is a skeleton of the code I'm trying to run to log traces in the phoenix UI

·

would you mind running this with llama index’s debug handler? this will tell us whether any traces are generated (which should print to console)

from llama_index.callbacks import CallbackManager, LlamaDebugHandler
from llama_index import ServiceContext

callback_handler = LlamaDebugHandler(print_trace_on_end=True)

service_context = ServiceContext.from_defaults(
    embed_model=embed_model,
    llm = llm,
    callback_manager=CallbackManager(handlers = [callback_handler])
)

Roger Y.
·
since we rely on Llama-Index’s callback system, the debug handler will signal whether the callback system is working
Samantha F.
·
Thanks Roger! I tried using the debug handler from llama-index and here's what it prints to the console:
Roger Y.
·
that looks good. thanks
Roger Y.
·
does the following code return any result?
session.get_spans_dataframe()?
Samantha F.
·
No it doesn't return anything
Roger Y.
·
ok got it. can you run it one more time using the phoenix handler, and then check the following result?
list(callback_handler.get_spans())
Roger Y.
·
it’s interesting that in your screenshot above shows CBEventType.LLM -> 0.0 seconds
Roger Y.
·
are you iterating on the streamed results?

·

what if you try it again but as a test try it with streaming=False

query_engine = index.as_query_engine(
                                similarity_top_k=4,
                                streaming=False)