Navigating Phoenix Tool for LLM Evaluations: Your Guide | Arize AI Community

Arize AI Community Icon

Navigating Phoenix Tool for LLM Evaluations: Your Guide | Arize AI Community

RunLLM
APP
·
To achieve detailed tracing and evaluation in Phoenix, you can follow these steps: 1. Trace Setup: Ensure your LangGraph application is properly instrumented for tracing. You can use Phoenix's tracing capabilities to capture spans, including retrieval spans, as described in the Phoenix tracing documentation.2. Dataset Integration: Import your CSV dataset into Phoenix. You can use the run_experiment function to load your questions and send them to your application. This will allow you to capture the responses and any associated retrieval spans.3. Evaluation: Use Phoenix's pre-built evaluators or create custom evaluators to assess the responses. Since you mentioned the absence of retrieved_documents, you might need to adjust your evaluation strategy. You can still evaluate the responses using LLM evaluators for correctness or relevance, even without ground truth, as Phoenix supports ad-hoc metric tracking on traces (source).Would you like more detailed guidance on any specific step? Sources: - https://github.com/Arize-ai/phoenix/issues/5274 - https://docs.arize.com/phoenix/retrieval/quickstart-retrieval#zOwQSPqp5axV
💡 Hint: Mention RunLLM in the thread for followups.
Yury Z.
·
Does Phoenix has a capability just send inputs to external API? In such case I can address questions directly to the application API, pull spans from Phoenix and run evaluations using 'run_eval'
Roger Y.
·
In your CSV file, are the questions the inputs to your langgraph application (which already has tracing)?
Roger Y.
·
e.g. my_langgraph_application.invoke(question)
Yury Z.
·
Roger Y. Yes. questions = inputs to langgraph application. And if I use chain.invoke(question) - I receive json with 'answer' but without retrieved_context
Yury Z.
·
But If I send the same question using application API => I see the chain with spans (including RETRIEVAL) into Phoenix
Roger Y.
·
i see. yea, run_experiment has the ability to feed the questions one by one to chain.invoke and the (internal) spans will show up in phoenix afterwards
Roger Y.
·
when you call run_experiment, there is a tank= parameter like this
experiment = run_experiment(dataset, task=task)
Roger Y.
·
that’s a function you can define like this
def task(input): return chain.invoke(input)
Roger Y.
·
what are the headers of your CSV file?
Roger Y.
·
you can try it out to see if it’s picking up the question correctly, if not we can tweak the input value
Roger Y.
·
there is also dry-run toggle if you just want to test it
experiment = run_experiment(dataset, task=task, dry_run=True)

·

def main(input) -> str:
    chain = create_chain()
    session_id_requests = f"TestBackend{random.randint(0, 1000000000)}"
    conversation_id_requests = f"TestBackendConversation{random.randint(0, 1000000000)}"
    upm_id_requests = f"TestBackendUpmId{random.randint(0, 1000000000)}"

    user_input = input["question"]
      
    data_input = {
                "language": "en",
                "marketplace": "US",
                "channel": "test_channel",
                "session_id": session_id_requests,
                "conversation_id": conversation_id_requests,
                "generated_text_response": False,
                "text": user_input,
                "upm_id": upm_id_requests,
                  }
    chain_out = chain.invoke(Request(**data_input))

    return chain_out.answer

I used this function as a task