Improving Arize Deployment UI Load Time for 550 Traces

Anuraag T. · 2024-08-27T16:12:05.127Z

Hi, In my arize deployment, it has lots of traces approx. 550 the issue I'm facing is that the UI takes 5-10 minutes to load.

RunLLM
APP
·
Phoenix's UI performance issues with large traces are known. Improvements include lazy loading of spans and reducing page size from 100 to 50 spans, which can help reduce load times (source). Would you like more details on how to implement these improvements? Sources: - GitHub Issue #4006
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
Hi Anuraag T., what version of Phoenix are you running and what pages are causing the issue?
Anuraag T.
·
It is deployed using azure app services(postgres, using the docker compose in github repo), version is 4.8.1. and its takes time loading project page.
Xander S.
·
Thanks Anuraag T.. Can you try upgrading to the latest version? We recently made some performance improvements on the project page.
Anuraag T.
·
Okay, will try upgrading to latest version. I have some other changes too in v4.8.1, would need to migrate that too.
Xander S.
·
Are you running a fork of Phoenix?
Anuraag T.
·
Not actually a fork, but something like that. added small patches to add traces from our mongodb.
Xander S.
·
I see.
Xander S.
·
I'm also curious about Jason's question regarding the number of spans. I'm also guessing that you're dealing with very large/ long context spans. Curious how many tokens your LLM calls are using?
Anuraag T.
·
I actually have 2 spans for each trace, 1 is ChatCompletion and 2 is Retriever span. so would be double of 550 approx. I also have a lot of text and documents in each span. token count of 1 trace is around 5k-7k, some times 8k.
Xander S.
·
Okay, thanks.
👍1
Xander S.
·
I expect upgrading Phoenix to the latest version to resolve this issue, but let us know if you continue to hit it.
Xander S.
·
Curious, what patch did you need to apply?
Anuraag T.
·
So the situation is, we have deployed our flask app through azure app services and it has some layers built on top of it which restricts us to use phoenix client direct to push spans using OTLP. To add a bypass(the patch I'm talking about) we're saving the spans to mongodb and then using the graphql query to fetch the data from mongodb and use BulkInserter to insert spans and evaluations into the postgre database.
Xander S.
·
I see. It's not possible to export OpenTelemetry traces in Azure app services?

19 comments