Hello Team, I've just started with phoenix. I'm using it for evaluations. I've created task function to generate response from my custom llm and using it with client.experiments.run_experiment(...) question is for repetitions=3, it is still processing 1 request at a time. Is there any way i can take advantage of threading with some x workers to generate concurrent response and make the process faster ? Thanks
Yes, you can run experiments with concurrent workers by setting the concurrency parameter in client.experiments.run_experiment. The repetitions parameter controls how many times each example is run, while concurrency determines how many tasks are processed in parallel. For example, set concurrency=5 to use 5 workers: LBKSexperiment = client.experiments.run_experiment( dataset=dataset, task=my_task, experiment_name="greeting-experiment", repetitions=3, concurrency=5 # Number of concurrent workers )LBKSAccording to the documentation, this will process multiple requests in parallel, making the experiment faster. For more, see How to: Experiments - Repetitions and phoenix-client docs.Would you like more detail on how concurrency interacts with repetitions or how to optimize for your custom LLM?Sources:
- https://github.com/Arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/index.md
- How to: Experiments - Repetitions
馃挕聽Hint: Mention RunLLM in the thread for followups.
with AsyncClient sys:1: RuntimeWarning: coroutine 'AsyncExperiments.run_experiment' was never awaited
run it with async function but still it was processing 1 request at a time
Client has no arg called concurrency
hi Darshan T., please use async_run_experiments
# Async experiments
from phoenix.client.experiments import async_run_experiment
from phoenix.client import AsyncClient
async def async_task(input):
return f"Hello {input['name']}"
async_client = AsyncClient()
dataset = await async_client.datasets.get_dataset(dataset="my-dataset")
experiment = await async_run_experiment(
dataset=dataset,
task=async_task,
experiment_name="greeting-experiment",
concurrency=5 # Run 5 tasks concurrently
) concurrency up to 40 is generally good!
SyntaxError: 'await' outside function if i wrap it around async function : httpcore.ConnectError: All connection attempts failed I verified Host and key is fine.
Darshan T. I copied your exact code and it worked for me. Are you locally hosting or cloud hosting? As a reminder, even though you said you checked host and api key, I want to make sure you have the correct source of truth. You can find your hostname and API Keys in the Phoenix UI -> settings -> General did your dataset get pulled? (dataset = await async_client.get_dataset...)
It is self-hosted
what is printed when you print host? also, did the dataset get pulled, or is that also failing?
Yes, it is able to pull the dataset. Host name is correct only
can you pass in the async_client into async_run_experiment
experiment = await async_run_experiment(
client=async_client,
dataset=dataset,
task=async_task,
experiment_name="sample_test",
concurrency=40
)馃敀[private user] yes that is working! Thanks one quick question. This concurrency is on response generation side (task method) or evaluation side or both?
both!
Cool! Let me play around it and again thanks for quick help!
of course! hope it works out well.
