Issue with Handler Activation in LlamaIndex Query Engine Setup

Adam K. · 2024-04-19T12:43:09.312Z

Hello all, I am seeing a weird issue. When I activate the handler in llamaindex: set_global_handler("arize_phoenix") etriever = VectorIndexRetriever( index=vector_index, similarity_top_k=10, ) # configure response synthesizer response_synthesizer = get_response_synthesizer( llm=llm, ) # assemble query engine query_engine = RetrieverQueryEngine( retriever=retriever, response_synthesizer=response_synthesizer, # node_postprocessors=[SimilarityPostprocessor(similarity_cutoff=0.7)], ) It keeps creating my spans like this:

Issue with Handler Activation in LlamaIndex Query Engine Setup | Arize AI Community

Adam K.
·
Adam K.
·
But this is wrong, it should be looking more like this:
Adam K.
·
where it is a chain i can dive into. I cannot even see the llm context prompt as well, so something is wrong.
Dustin N.
·
Hi Adam, thanks for your interest in Phoenix! There’s currently an issue with llamaindex instrumentation where sometimes slans aren’t grouped correctly due to changes in their callback system, we’re currently working with the their team to get this resolved on their end. Meanwhile there can be several causes for this, the spans might not group until the root span is closed, if you’re producing streaming results, can you try consuming all the streams? Sometimes it will group the spans then
Adam K.
·
Thanks Dustin N. - so it seems that if I do the QueryPipeline approach instead of my manual way I showed above, it groups it correctly.
Adam K.
·
Adam K.
·
writing code like this groups them
Dustin N.
·
No problem! Glad to hear that’s working for now. We’re hoping to resolve this soon
Adam K.
·
Dustin N. unfortunately I cannot use this a solution, have you guys fixed the issue with grouping the spans because it makes tracking and debugging useless.
Dustin N.
·
Adam K. darn :(. We’re hoping to fix this as soon as possible, though last I checked we’re waiting on some updates on LlamaIndex as well that they’re actively working on. There are manual instrumentation approaches that might help resolve the issue temporarily, is there any chance you can give us a MRE to look at?
Adam K.
·
Hey Dustin N. thanks for the reply. Can you give me a hint as to how to do it manually, I can try that. What is MRE? Meal , ready to eat? lol
Dustin N.
·
Adam K. sorry about that! Here’s our tutorial for manual instrumentation: https://github.com/Arize-ai/phoenix/tree/main/examples/manually-instrumented-chatbot. I’m hoping depending on what your setup looks like we can manually add a span that ties everything together for you. By an MRE I mean a minimal reproducible example that we can use to test out
Adam K.
·
that looks pretty painful to add all that manually
Dustin N.
·
Yeah, the hope is we only need to add one so your traces can be grouped together, it'd hard for us to tell for sure without an example to think about though

Adam K.

Dustin N. the MRE is really something like this:

set_global_handler("arize_phoenix")
query_engine = vector_index.as_query_engine(
    llm=llm,
    similarity_top_k=20,
    node_postprocessors=[reranker],
    refine_template=PromptTemplate(prompt_str),
    # text_qa_template=PromptTemplate(prompt_str),
)
output = query_engine.query("my query")

24 comments