Ben M.

I will end up doing something like that, but it might be neat if we augmented the tracer provider with a context manager. Something like this

with tracer_provider.flush_context():
    # call models, etc

Then when the context is exited, it can automatically force flush.

Thanks for the heads up 👍

Does it make sense for that to be part of phoenix.otel? I understand if not, but am open to submitting a PR if you think it would be more widely beneficial 🙂

Thanks, Roger. I’ll probably just end up hooking into atexit to force flush. https://docs.python.org/3/library/atexit.html

Emanuele B. is this what you’re looking for? https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-google-adk Phoenix is just an openinference collector, so the compatibility question is actually with the instrumentation package 🙂

I may end up just using httpx to do this via the API for now. It’s not that much code in the grand scheme of things

That does get me 1 step closer. I would like to get to a point where we can also run individual tasks in the client. Meaning, I would love to be able to pull my dataset and also scale out the actual task completions and logging results. I put together a quick mermaid diagram to show. I’d love each cell to be able to be its own lambda function, reading from the queue and asynchronously building up our outputs and then once generated running all of our evals in parallel in other cloud functions. I think the function you shared lets me do that last step but I need to figure out if the client has something for taking an example by id and logging them to an experiment run

https://github.com/Arize-ai/phoenix/issues/9907

You got it!