Troubleshooting OpenInference Tracing in Arize with Pytest

Samantha Z. · 2025-02-10T19:11:54.151Z

I'm trying to set up openinference tracing, and I would expect to see something show up in my dashboard when I run my tests, but I don't see my new project being created. Does Arize automatically skip tracing when code is being run from e.g. pytest? Or do I need to do something else to enable it?

Troubleshooting OpenInference Tracing in Arize with Pytest | Arize AI Community

RunLLM
APP
·
Arize does not automatically skip tracing when code is run from testing frameworks like pytest. To ensure that your traces are captured and sent to the Phoenix dashboard, you need to verify a few things: 1. Tracer Initialization: Ensure that the tracer is correctly initialized before any code that you want to trace is executed. This includes setting up the tracer provider and span processor as shown in the Phoenix documentation.2. Environment Variables: Make sure that the necessary environment variables, such as PHOENIX_COLLECTOR_ENDPOINT, are correctly set. This endpoint should point to your Phoenix server, for example, http://localhost:6006/v1/traces.3. Project Configuration: Ensure that the project name is correctly set in your tracing setup. Projects in Phoenix are created when traces are sent with a new project name. You can set the project name using the register function from the phoenix.otel module, as described in the setup guide.Would you like more detail on any specific part of this setup?Sources: - Phoenix documentation
💡 Hint: Mention RunLLM in the thread for followups.
Samantha Z.
·
I already did all of those things. That's why I'm asking here.
Samantha Z.
·
I would've expected to see a 'hello world' example to make sure tracing is hooked up as expected.
Mikyo
·
Hey Samantha Z. - sorry for the dumb question but is the code that is run during your testing (e.g. pytest) actually initialize opentelemetry?

Mikyo

·

As in you will have to have something like

import os
from phoenix.otel import register

# configure the Phoenix tracer
tracer_provider = register(
  project_name="my-pytest", 
)

at the beginning of the tests

Samantha Z.
·
That's what I wanted to know, if we just have it in our main code, and the tests call that code, is that good enough? Sounds like it's not.
Samantha Z.
·
Thanks!
Mikyo
·
If you run experiments as part of CI (https://docs.arize.com/phoenix/datasets-and-experiments/how-to-experiments/run-experiments_) then the setup is done for you automatically based on env vars but if you are rolling your own tests it would need some setup code to setup tracing. Hope that makes sense
Samantha Z.
·
This is still not working. I'm also struggling to find the instructions on how to run a local server vs. using the Arize endpoint.
Mikyo
·
you can run phoenix locally by just running phoenix serve and that will run a local phoenix instance using sqlite or what's configured in your environment variables. Could you post a code snippet of what a test looks like for you?
Mikyo
·
Hey Samantha Z. sorry we didn't get to a resolution later. Happy to help in any way. Copying my colleague John G. - let us know if there's anything we can do!
John G.
·
Samantha Z. - let us know if it would be helpful to jump on a call and debug live!
Samantha Z.
·
Thanks! I think I need to try it one more time, and in the meantime we have a shared self-hosted version that we should be able to use, so I'm going to try that, also
🙌1
John G.
·
Okay sounds good! Keep us in the loop and shout if you want some live help
❤️1
Samantha Z.
·
ok I got it working. Turned out it was a problem with my virtual environment configuration. yay! Thanks for helping me check my work.
🙌1

26 comments