Future Plans for Supporting Datasets as API Payloads in Phoenix

·Jul 08, 2025 08:27 PM

Another thing I’d like to ask: does this project ever plan on supporting running datasets as payloads to arbitrary APIs in the future? My LLM agents do complex things and I’d love to let my business analysts create data sets in Phoenix and click a “test run” type of button on all of them on a live API. I know I could set this up myself using the Python API - but would love to do it natively. I’d also be very interested in contributing a pull request to help develop this feature.

2 comments

· Sorted by Oldest

RunLLM
APP
·
Phoenix does not currently support running datasets as payloads to arbitrary APIs natively via the UI or a "test run" button; this workflow must be set up manually using the Python API or client. However, this feature has been requested by users, and the Phoenix team welcomes contributions—especially for enhancements like this. You are encouraged to start with a "good first issue" on the GitHub board and submit a pull request for new features or improvements. The team reviews PRs promptly and provides feedback for approval and merging.If you want to contribute, fork the Phoenix repository, create a new branch, and open a PR describing your changes and motivation. For more details, see the contribution guidelines and development guide in the documentation. Would you like more detail on the current experiment/dataset API or guidance on where to start your contribution?Sources: - https://github.com/Arize-ai/phoenix/issues/7528 - https://arize-ai.slack.com/archives/C04R3GXC8HK/p1751374792.815499
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
Hey Graham R., we do support an experiments API that allows you run arbitrary code (including invoking an agent). Not sure if that is what you have in mind. https://arize.com/docs/phoenix/datasets-and-experiments/overview-datasets
:thankyou:1