Issues After Upgrading Arize and Llama Index Libraries

alex · 2025-01-31T01:00:55.258Z

Recently I upgraded the following requirements arize-phoenix==4.20.2 > 7.9.2 llama-index-core==0.10.44 > 0.12.7 llama-index-callbacks-arize-phoenix==0.1.5 > 0.3.0 What I noticed was 1. In my collapsed span of traces, previously I would see MY_TITLE but now it's all llama index methods. Is the title or I guess QUERY_STR deprecated? with callback_manager.event( CBEventType.RETRIEVE, payload={EventPayload.QUERY_STR: "MY_TITLE"}, 2. When looking with in the phoenix spans of reranked nodes Previously I could I see the text/content of the reranked nodes. Now I just see the reranked nodes ids without text, so I can't really tell how the content changed unless I compare the initially retrieved nodes, and the final synthesizer part which has context of the reranked nodes. I'm not sure if this is a llama index or arize thing, so thought would ask here.

Issues After Upgrading Arize and Llama Index Libraries | Arize AI Community

7 comments

· Sorted by Oldest

RunLLM
APP
·
1. QUERY_STR Deprecation: The QUERY_STR payload might have been deprecated or changed in the recent updates. The llama-index and arize-phoenix updates could have altered how events and payloads are handled, leading to the change in how titles or queries are displayed in the collapsed span of traces. It's possible that the llama-index methods are now more prominently displayed due to changes in the tracing or callback handling.2. Reranked Nodes Text: The issue with not seeing the text/content of reranked nodes might be related to changes in how llama-index or arize-phoenix handles the retrieval and display of document content. The recent updates might have affected the way document content is stored or retrieved in the spans, leading to only node IDs being visible. This could be due to changes in the callback system or how the llama-index processes and stores document information.Would you like more detail on these points?Sources: - https://github.com/Arize-ai/phoenix/issues/2047 - https://github.com/Arize-ai/phoenix/issues/2218
💡 Hint: Mention RunLLM in the thread for followups.

Roger Y.

can you try running this at the top of your program?

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider  
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))

LlamaIndexInstrumentor().instrument(
    tracer_provider=tracer_provider,
    use_legacy_callback_handler=True,
)

alex
·
ok thanks. let me try it and get back to you
👍1
alex
·
Roger Y. thanks this works!
Roger Y.
·
great!
alex
·
whats the benefit to the new way vs the legacy way from a llama index perspective.
Roger Y.
·
it’s a new system that’s more generic in the sense that it’s implemented as decorator on any function, and it’s supplemented with a new system of events, whereas the previous system was just mostly ad hoc dictionaries and are less well typed
🙏1

RunLLM
APP
·
1. QUERY_STR Deprecation: The QUERY_STR payload might have been deprecated or changed in the recent updates. The llama-index and arize-phoenix updates could have altered how events and payloads are handled, leading to the change in how titles or queries are displayed in the collapsed span of traces. It's possible that the llama-index methods are now more prominently displayed due to changes in the tracing or callback handling.2. Reranked Nodes Text: The issue with not seeing the text/content of reranked nodes might be related to changes in how llama-index or arize-phoenix handles the retrieval and display of document content. The recent updates might have affected the way document content is stored or retrieved in the spans, leading to only node IDs being visible. This could be due to changes in the callback system or how the llama-index processes and stores document information.Would you like more detail on these points?Sources: - https://github.com/Arize-ai/phoenix/issues/2047 - https://github.com/Arize-ai/phoenix/issues/2218
💡 Hint: Mention RunLLM in the thread for followups.

Roger Y.

can you try running this at the top of your program?

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider  
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

endpoint = "http://127.0.0.1:6006/v1/traces"
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))

LlamaIndexInstrumentor().instrument(
    tracer_provider=tracer_provider,
    use_legacy_callback_handler=True,
)

alex
·
ok thanks. let me try it and get back to you
👍1
alex
·
Roger Y. thanks this works!
Roger Y.
·
great!
alex
·
whats the benefit to the new way vs the legacy way from a llama index perspective.
Roger Y.
·
it’s a new system that’s more generic in the sense that it’s implemented as decorator on any function, and it’s supplemented with a new system of events, whereas the previous system was just mostly ad hoc dictionaries and are less well typed
🙏1