Yes, you can run experiments on a subset of examples using the dry_run parameter in both run_experiment() and evaluate_experiment(). Setting dry_run=True runs one sample, or dry_run=N runs N randomly selected examples, allowing for quick, deterministic testing without processing the full dataset or sending data to the Phoenix server. This is the recommended approach for faster iteration on a subset of your data.
For more advanced filtering (e.g., by label or metadata), you would need to pre-filter your dataset before passing it to Phoenix, as there is no built-in experiment API for arbitrary subset selection beyond dry_run. Would you like more detail on how to implement this?
Sources:
- <https://arize.com/docs/phoenix/datasets-and-experiments/quickstart-datasets#typescript-8|Phoenix Experiments Quickstart: Dry Run>
- <https://arize.com/docs/phoenix/datasets-and-experiments/how-to-experiments/run-experiments#«Rdq6jtt9lffffffb»|Phoenix Experiments How-to: Dry Run>
- <https://github.com/Arize-ai/phoenix/blob/main/docs/datasets-and-experiments/how-to-experiments/run-experiments.md|Phoenix Experiments GitHub Docs: Dry Run>
