Troubleshooting Langchain Query Evaluation in Phoenix UI

·Feb 27, 2024 03:49 AM

hi team, i have some issue when evaluating langchain query, when i run this code (which i follow from the notebook tutorial) def evaluation(self): queries_df = get_qa_with_reference(Client()) retrieved_documents_df = get_retrieved_documents(Client()) it always evaluate all the queries in my Phoenix UI, I've tried filter using span_id but not working, any idea how can I just evaluate one query instead of all in the UI?

6 comments

· Sorted by Oldest

Mikyo
·
Hey Anastasia B. can you elaborate on the span_id based filtering that didn't work?

Anastasia B.

since that didn't work, i try another method with WHERE, and modify that in get_retrived_documents function like this,

def get_retrieved_documents(obj: CanQuerySpans, query: str) -> pd.DataFrame:
    words = query.split()
    return cast(
        pd.DataFrame,
        obj.query_spans(
            SpanQuery()
            .where(IS_RETRIEVER)
            .where(word in words for word in words)  
            .select("trace_id", **INPUT)
            .explode(
                RETRIEVAL_DOCUMENTS,
                reference=DOCUMENT_CONTENT,
                document_score=DOCUMENT_SCORE,
            )
        ),

but still got error when running

return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: compile() arg 1 must be a string, bytes or AST object

Mikyo
·
The error is telling you that the following is not correct - (it needs to be a string or an AST object) word in words for word in words
Mikyo
·
Check out the query DSL and let us know if that works! https://docs.arize.com/phoenix/how-to/extract-data-from-spans
Anastasia B.
·
hey, it's already work, thanks! but i just curious, is there any other way to filter query e.g using user id if any or else?
Mikyo
·
You can filter by anything in the attributes, but things like user id would have to be sent in via custom attributes or metadata https://docs.arize.com/phoenix/tracing/custom-spans