Getting Started with Root Cause Analysis in Arize Documentation

Roxana L. · 2024-07-03T15:55:55.505Z

Hi all, I'm new to Arize, I see that it does root cause analysis ("Trace the root cause back to problematic data") can you point me to the specific portion of the docs that explains what the capability entails?

Getting Started with Root Cause Analysis in Arize Documentation | Arize AI Community

8 comments

· Sorted by Oldest

Tammy L.
·
Hi Roxana - glad you're trying out Arize. RCA or tracing in the Arize platform depends on your particular use case. The net of it is that we provide monitoring that first helps detect a performance regression, then workflows that help you drill into the specifics of where in your data that issue is occurring -- e.g. broken LLM retrieval, a new model feature, etc. Here are docs that you can look at to learn more about the various options:
LLM tracing
ML performance tracing
ML drift tracing
ML bias tracing
Embeddings analyzer
Roxana L.
·
Thank you Tammy, is the RCA on Arize "true" RCA as in telling me the beginning of a chain of things that derailed my model performance? Not talking about LLMs here. Also, is the RCA limited to low-performing cohort analysis?
Jason
·
Roxana L. If you have ground truth and performance is a focus: Lots of tools to trace to poor Performance Cohorts from estimated cohorts of problems, embeddings cluster workflows if you are working with data CV/NLP/Tabular with embeddings. if you don't have ground truth: drift and data quality tools help trace to where data is possibly causing issues. The drift tracing attempts to track output changes connections to input data movements. There are a lot of data quality analysis tools per feature as well for catching NULLs or data movements on features.
Tammy L.
·
This video might be helpful to understand a flow you might go through as you perform RCA. We have a concept of "worst performing slices or clusters" that provide an intuitive launch point to kickoff your investigation: https://www.youtube.com/watch?v=pmCZIifVQ0o
Roxana L.
·
so I still have to to RCA myself, Arize just gives me pointers and directions to search, but not the answer
Tammy L.
·
There are a few product launches that we're announcing at our conference, Observe, next week - worth catching, as it'll address what you're after here
Roxana L.
·
Really? Sounds awesome! I don't have tickets, will you announce the new features anywhere?
Tammy L.
·
Yep! We'll share the announcements through the usual channels, including product release notes
🙌1

Tammy L.
·
Hi Roxana - glad you're trying out Arize. RCA or tracing in the Arize platform depends on your particular use case. The net of it is that we provide monitoring that first helps detect a performance regression, then workflows that help you drill into the specifics of where in your data that issue is occurring -- e.g. broken LLM retrieval, a new model feature, etc. Here are docs that you can look at to learn more about the various options:
LLM tracing
ML performance tracing
ML drift tracing
ML bias tracing
Embeddings analyzer
Roxana L.
·
Thank you Tammy, is the RCA on Arize "true" RCA as in telling me the beginning of a chain of things that derailed my model performance? Not talking about LLMs here. Also, is the RCA limited to low-performing cohort analysis?
Jason
·
Roxana L. If you have ground truth and performance is a focus: Lots of tools to trace to poor Performance Cohorts from estimated cohorts of problems, embeddings cluster workflows if you are working with data CV/NLP/Tabular with embeddings. if you don't have ground truth: drift and data quality tools help trace to where data is possibly causing issues. The drift tracing attempts to track output changes connections to input data movements. There are a lot of data quality analysis tools per feature as well for catching NULLs or data movements on features.
Tammy L.
·
This video might be helpful to understand a flow you might go through as you perform RCA. We have a concept of "worst performing slices or clusters" that provide an intuitive launch point to kickoff your investigation: https://www.youtube.com/watch?v=pmCZIifVQ0o
Roxana L.
·
so I still have to to RCA myself, Arize just gives me pointers and directions to search, but not the answer
Tammy L.
·
There are a few product launches that we're announcing at our conference, Observe, next week - worth catching, as it'll address what you're after here
Roxana L.
·
Really? Sounds awesome! I don't have tickets, will you announce the new features anywhere?
Tammy L.
·
Yep! We'll share the announcements through the usual channels, including product release notes
🙌1