Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Understanding NDCG Score and Metrics for Retriever Documents | Arize AI Community

Phoenix Support

Understanding NDCG Score and Metrics for Retriever Documents

·Sep 10, 2024 04:16 PM

Hi, How is the ndcg score and other metrices for retriever documents are calculated what is the logic behind them?

5 comments

· Sorted by Oldest

RunLLM
APP
·
The NDCG (Normalized Discounted Cumulative Gain) score and other metrics for retrieved documents are calculated based on the relevance scores of the documents. Here's a brief overview: 1. NDCG: Measures the ranking quality of retrieved documents, considering the position of relevant documents. Higher scores indicate better ranking. The formula involves discounting the relevance scores logarithmically based on their positions. For more details, see RetrievalMetrics.ndcg.2. Precision: Fraction of relevant documents among the retrieved ones. Higher precision means more relevant documents were retrieved. For more details, see RetrievalMetrics.precision.3. Hit: Indicates if at least one relevant document was retrieved. A hit rate of 1 means at least one relevant document was found. For more details, see RetrievalMetrics.hit.Would you like more detail?Sources: - RetrievalMetrics.ndcg
💡 Hint: Mention RunLLM in the thread for followups.
Roger Y.
·
the bot is pointing the correct location above: you can check out the link shown for more info
Roger Y.
·
ah the webpage is offline right now
👍1
Roger Y.
·
oh it’s back https://cran.r-project.org/web/packages/recometrics/vignettes/Evaluating_recommender_systems.html
Anuraag T.
·
thanks!
👍1