Evaluating a Subset of Dataset with run_experiment Function

Zhuyi X. · 2025-07-09T23:15:55.036Z

when using run_experiment, is there to only evaluate on a subset of a dataset (e.g. pick a few examples by id)?

Phoenix does not provide a built-in parameter in run_experiment to directly select a subset of examples by ID. However, you can create a new dataset containing only the desired examples and run the experiment on that subset, or use metadata/labels to filter and construct the subset before running the experiment. There is no explicit API for filtering by example ID at experiment runtime in the provided data sources.Would you like more detail on how to create a filtered dataset or use metadata for this purpose?Sources: - Phoenix Slack Discussion

💡 Hint: Mention RunLLM in the thread for followups.

3 comments