Tips to Speed Up Slow Search Sessions in Queries | Arize AI Community

Arize AI Community Icon

Tips to Speed Up Slow Search Sessions in Queries | Arize AI Community

RunLLM
APP
·
It seems likely that searching sessions by message content in Phoenix can be slow, especially as data size grows, due to how the backend queries are structured and the lack of full-text indexing or optimizations for large attributes. There are known reports of slow GraphQL queries and suggestions to avoid pulling large attributes over connections, as this can significantly impact performance—even with moderate data sizes. Some improvements have been made (e.g., reducing page size, lazy loading), but searching message content remains a bottleneck if the session or span attributes are large or not indexed for text search. My understanding is that, as of now, there are no user-configurable settings to speed up this specific search, and the slowness is a recognized limitation in the current implementation (source, source, source).Possible workarounds mentioned include limiting the size of stored messages, using previews instead of full content, or offloading large messages to blob storage and storing only references in Phoenix. There is no mention in the data sources of a direct way to enable full-text search or indexing for session messages, nor a configuration to speed up these queries in the UI or backend. Would you like more detail on the technical reasons for this slowness or on potential architectural workarounds?Sources: - github.com/Arize-ai/phoenix/issues/4006 - arize-ai.slack.com/archives/C04R3GXC8HK/p1718305854.864919 - https://github.com/Arize-ai/phoenix/pull/5204
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
Hey Zhuyi X., can you provide a screenshot of the page you are on and the query you are running? It sounds like you are on the sessions page?
Zhuyi X.
·
yeah, I was on sessions page like
Zhuyi X.
·
this one is mroe representative, it says 5.7M tokens over last day.
👍1
Zhuyi X.
·
I just typed some text on the Search messages box, the browser appears to do nothing, not returning anything till 53s later (I get the number from the chrome developer console)
Zhuyi X.
·
the query is a short text with ~10 words
Xander S.
·
Zhuyi X. Can you tell me more about the nature of your data? For example, do you have particularly long messages?
Zhuyi X.
·
it's basically a chatbot
Zhuyi X.
·
like LLM+a bunch of custom built tools
Zhuyi X.
·
what's being traced is very similar to what's introduced in this course https://www.deeplearning.ai/short-courses/evaluating-ai-agents/
Zhuyi X.
·
agent => router call => tool call => LLM calls.
Xander S.
·
Okay, so it doesn't sound like the size of the messages is particularly long.
Xander S.
·
Out of curiosity, are you using Phoenix Cloud or self-hosting?
Zhuyi X.
·
does the search search span metadata? in the metadata, some the spans may have logged some opaque tokens (long strings)
Zhuyi X.
·
self-hosted
👍1