OpenInference Instrumentation: Logging Tools & Package Compatibility

Zhuyi X. · 2025-06-25T16:46:34.835Z

is it the case that openinfernece-instrumentation-openai-agents doesn't log tools by default? E.g. comparing the auto instrumentation output between openinfernece-instrumentation-openai-agents (left) and openinfernece-instrumentation-openai (right) also, is it a good idea to use the two packages together?

8 comments

· Sorted by Oldest

RunLLM
APP
·
By default, `openinference-instrumentation-openai-agents` does log tools, including agent workflows, LLM calls, tool usage, and handoffs, when instrumented properly. It provides enhanced visibility into agent behavior compared to `openinference-instrumentation-openai`, which only traces OpenAI SDK calls and not agent-level workflows or tool usage. Using both instrumentors together is not recommended, as it can lead to duplicate or conflicting traces; instead, use only the agents instrumentor for agent-based applications to ensure correct and comprehensive tracing of tools and workflows. This is confirmed by the official integration and troubleshooting guidance, which shows that disabling agent tracing causes traces to revert to basic OpenAI call traces, losing agent/tool context, and that the agents instrumentor is designed to capture tool usage by default when set up correctly with `exclusive_processor=True` or `auto_instrument=True` in Phoenix's `register` function (<https://github.com/Arize-ai/openinference/issues/1686|source>, <https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation|source>). Would you like a step-by-step example of how to set up tracing for agent tools with the correct instrumentor? Sources: - <https://github.com/Arize-ai/openinference/issues/1686|https://github.com/Arize-ai/openinference/issues/1686> - <https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation|https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation>
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
hi Zhuyi X. I believe the agents instrumentor should log tools by default, and both the agents instrumentor and the sdk instrumentor compose well (you can use both at once)
1
Zhuyi X.
·
agents instrumentor should log tools by default
is this configured in instrumentor lib or openai-agents sdk?
Dustin N.
·
you shouldn't have to configure anything
Zhuyi X.
·
hmm, then it's kind of odd why tools aren't logged as in the screenshot to the left. here is a more complete picture, as seen, the input has no tools, but the output is a tool call, is that expected? maybe it's related to me using OpenAIChatCompletionsModel like in https://github.com/openai/openai-agents-python/blob/c2005f8273a899ffd32e4b77c06701f347f8ec92/examples/model_providers/custom_example_agent.py#L46?
Zhuyi X.
·
I'll just use the two instrumentors together for now as the sdk instrumentor logs tools
Dustin N.
·
Zhuyi X. thanks for noting this, it's definitely possible we're failing to capture certain tools with the agent instrumentor, if you have a reproducible example please file an issue and we'll check it out!
Zhuyi X.
·
added https://github.com/Arize-ai/openinference/issues/1800
1

RunLLM
APP
·
By default, `openinference-instrumentation-openai-agents` does log tools, including agent workflows, LLM calls, tool usage, and handoffs, when instrumented properly. It provides enhanced visibility into agent behavior compared to `openinference-instrumentation-openai`, which only traces OpenAI SDK calls and not agent-level workflows or tool usage. Using both instrumentors together is not recommended, as it can lead to duplicate or conflicting traces; instead, use only the agents instrumentor for agent-based applications to ensure correct and comprehensive tracing of tools and workflows. This is confirmed by the official integration and troubleshooting guidance, which shows that disabling agent tracing causes traces to revert to basic OpenAI call traces, losing agent/tool context, and that the agents instrumentor is designed to capture tool usage by default when set up correctly with `exclusive_processor=True` or `auto_instrument=True` in Phoenix's `register` function (<https://github.com/Arize-ai/openinference/issues/1686|source>, <https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation|source>). Would you like a step-by-step example of how to set up tracing for agent tools with the correct instrumentor? Sources: - <https://github.com/Arize-ai/openinference/issues/1686|https://github.com/Arize-ai/openinference/issues/1686> - <https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation|https://arize.com/docs/phoenix/release-notes/03.14.2025-openai-agents-instrumentation>
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
hi Zhuyi X. I believe the agents instrumentor should log tools by default, and both the agents instrumentor and the sdk instrumentor compose well (you can use both at once)
1
Zhuyi X.
·
agents instrumentor should log tools by default
is this configured in instrumentor lib or openai-agents sdk?
Dustin N.
·
you shouldn't have to configure anything
Zhuyi X.
·
hmm, then it's kind of odd why tools aren't logged as in the screenshot to the left. here is a more complete picture, as seen, the input has no tools, but the output is a tool call, is that expected? maybe it's related to me using OpenAIChatCompletionsModel like in https://github.com/openai/openai-agents-python/blob/c2005f8273a899ffd32e4b77c06701f347f8ec92/examples/model_providers/custom_example_agent.py#L46?
Zhuyi X.
·
I'll just use the two instrumentors together for now as the sdk instrumentor logs tools
Dustin N.
·
Zhuyi X. thanks for noting this, it's definitely possible we're failing to capture certain tools with the agent instrumentor, if you have a reproducible example please file an issue and we'll check it out!
Zhuyi X.
·
added https://github.com/Arize-ai/openinference/issues/1800
1