Maintaining Retriever Spans with SubQuestionQueryEngine in LlamaIndex

Delon S. · 2024-01-05T20:14:31.606Z

Hey all, does anyone know how to keep the retriever spans available when using a SubQuestionQueryEngine in LlamaIndex?

Maintaining Retriever Spans with SubQuestionQueryEngine in LlamaIndex | Arize AI Community

Roger Y.
·
Hi Delon. Let me look into that for you.
Roger Y.
·
Are you referring to these spans in the screenshot below? I got these from running the example notebook from LlamaIndex.
Delon S.
·
Hi Roger, yes I am
Delon S.
·
as well as the Reranker spans
Roger Y.
·
are you not seeing thees spans in Phoenix when you tried it?
Delon S.
·
with certain query engines (llamaindex abstraction) but not others, even though I am assigning the same callbacks
Delon S.
·
the SubQuestionQueryEngine, for example, doesn't show them
Delon S.
·
whereas a RetrieverQueryEngine does
Roger Y.
·
I see. We have actually encountered a similar problem with spans not showing up when the query engines is set up differently. This is due to the fact that we rely on LlamaIndex callbacks, but sometimes we just don’t receive the event from LlamaIndex

Roger Y.

you can try the debug callback manager from LlamaIndex

from llama_index.callbacks import CallbackManager, LlamaDebugHandler
from llama_index import ServiceContext

llama_debug = LlamaDebugHandler(print_trace_on_end=True)
callback_manager = CallbackManager([llama_debug])
service_context = ServiceContext.from_defaults(
    callback_manager=callback_manager
)

Roger Y.

it’s supposed to print out the events like this

**********
Trace: query
    |_query ->  13.064431 seconds
      |_llm ->  2.499768 seconds
      |_sub_question ->  2.05934 seconds
        |_query ->  2.059142 seconds
          |_retrieve ->  0.278184 seconds
            |_embedding ->  0.274593 seconds
          |_synthesize ->  1.780895 seconds
            |_llm ->  1.740488 seconds
      |_sub_question ->  5.364061 seconds
        |_query ->  5.363695 seconds
          |_retrieve ->  0.230257 seconds
            |_embedding ->  0.226763 seconds
          |_synthesize ->  5.133343 seconds
            |_llm ->  5.091069 seconds
      |_sub_question ->  2.148964 seconds
        |_query ->  2.14889 seconds
          |_retrieve ->  0.323438 seconds
            |_embedding ->  0.319841 seconds
          |_synthesize ->  1.825401 seconds
            |_llm ->  1.783064 seconds
      |_synthesize ->  5.198214 seconds
        |_llm ->  5.175849 seconds
**********

Roger Y.
·
if it’s not printing out the event that you’re looking for, we probably don’t receive it in Phoenix
Delon S.
·
okay got it, the print statement you showed is from running a SubQuestionQueryEngine on your end? The implementation i'm dealing with is also part of a chat engine
Roger Y.
·
yes, it’s from that notebook i linked to above. when i ran that notebook, the retrieve spans do show up in Phoenix
Delon S.
·
ah interesting okay

17 comments