Comparing Effectiveness of LLM-eval vs Non-LLM-Eval Techniques

·Jun 22, 2024 08:39 AM

Hello, can someone throw light on effectiveness of LLM-eval and Non-LLM-Eval? Which is better currently? If there are any web articles / research papers regarding it please share it?

Phoenix Support

Comparing Effectiveness of LLM-eval vs Non-LLM-Eval Techniques

Juswanth ..

·Jun 22, 2024 08:39 AM

Hello, can someone throw light on effectiveness of LLM-eval and Non-LLM-Eval? Which is better currently? If there are any web articles / research papers regarding it please share it?

4 comments

· Sorted by Oldest

Nabeegh A.
·
Hello - This entirely depends on your use case. What kind of data do you generate, what range of inputs and outputs do you except. What format the data follows. All of these questions determine which one you want to use. It is not a question of which one is better. Both of them have their own use cases.
Juswanth ..
·
I want to evaluate RAG pipeline using different metrics. Can you suggest some of the best ways for it?
Nabeegh A.
·
Since the data generated and inputs given to the system are very diverse, you cannot use a fixed set of rules to evaluate the system. For evaluating RAG pipelines you need to use LLM-Evals. Phoenix already has a Retrieval (RAG) Relevance evaluation template available that you can use. You can also check out RAGAS.
Juswanth ..
·
Got it, thanks for the insights guys 🙏

Nabeegh A.
·
Hello - This entirely depends on your use case. What kind of data do you generate, what range of inputs and outputs do you except. What format the data follows. All of these questions determine which one you want to use. It is not a question of which one is better. Both of them have their own use cases.
Juswanth ..
·
I want to evaluate RAG pipeline using different metrics. Can you suggest some of the best ways for it?
Nabeegh A.
·
Since the data generated and inputs given to the system are very diverse, you cannot use a fixed set of rules to evaluate the system. For evaluating RAG pipelines you need to use LLM-Evals. Phoenix already has a Retrieval (RAG) Relevance evaluation template available that you can use. You can also check out RAGAS.
Juswanth ..
·
Got it, thanks for the insights guys 🙏