Implementing a Custom Document Evaluator in Phoenix

Is there a way to implement a custom evaluator that doesn't use an LLM? One way I want to evaluate retrieval is whether a certain document appears in the results for a certain query. I have a dataset of query --> expected document to use for evaluation, and want to run retrieval for the queries in this dataset and measure how often the results include the expected document, and score based on what position the expected document was found. How would I do this using phoenix and attach this custom eval to a span?