hey folks, is it correct that experiment names must be unique, even if they’ve been deleted? if I understand the intent properly, we’d want to run them in a ci/cd pipeline, ie. we’d essentially be running the same experiments over and over. are folks typically appending a timestamp, or a build ID or something? it would be handy to be able to version them, similar to datasets, rather than requiring a new one to always be created. or some other way to organize them, ie. by metadata tags, or folders. as it stands, I imagine the experiments page is going to get pretty unwieldy.
Hi Adam! Good to see you here 🙂 Yes you're correct. The experiment names must be unique. Just tested your flow, and it does throw an error even if the experiment is deleted. But I think you're right that if you delete the experiment, it doesn't make sense to still require the experiment name to be unique. Will file a ticket to fix that. Teams typically do append a timestamp or build ID so that they version the experiment, but it's a good note that we should probably add support to handle this. We are going to be adding supporting for metadata tags on experiments soon so that teams can quickly filter through them!
Oh, one other thing I noticed… there’s nothing documented in the Python client API (https://arize-client-python.readthedocs.io/en/latest/llm-api/datasets.html#arize.experimental.datasets.ArizeDatasetsClient) to delete an experiment, although you can do it from the UI. Once your ticket gets fixed and the experiment will get properly deleted, then I could see doing this programmatically when re-running an experiment (although versioning it would probably be better than deleting and re-creating).
