Understanding Span Kind Chain vs. Agent in LLM Instrumentation

·Mar 19, 2025 11:38 AM

Hi, would like to ask about the span kind chain vs. agent - sometimes I realize the anthropic llm span isn't automatically instrumented for under chain but works fine with agent - what could be the reason? Thanks! 🙏

9 comments

· Sorted by Oldest

RunLLM
APP
·
It seems likely that the difference in behavior between `chain` and `agent` span kinds could be due to how these spans are defined and used in Phoenix. According to the <https://docs.arize.com/phoenix/~/changes/v6Zhm276x8LlKmwqElIA/reference/open-inference#B%3A14|OpenInference Tracing documentation>, a `CHAIN` span is typically used for general logic operations or code blocks, while an `AGENT` span is used for invocations that encompass calls to LLMs and tools, often acting as a reasoning block. This distinction might affect how spans are automatically instrumented, especially if the `agent` span is designed to better capture the orchestration of LLM and tool interactions. Additionally, the <https://github.com/Arize-ai/phoenix/issues/1311|Phoenix GitHub issue> suggests that the `agent` span kind was introduced to delineate from `chain` because the attributes for this span kind are different. This could mean that the `agent` span is more suited for capturing complex interactions involving LLMs, which might explain why the `llm` span works better under `agent`. Would you like more detail on how to configure these spans or troubleshoot further? Sources: - <https://github.com/Arize-ai/phoenix/issues/1311|Phoenix GitHub issue> - <https://docs.arize.com/phoenix/~/changes/v6Zhm276x8LlKmwqElIA/reference/open-inference#B%3A14|OpenInference Tracing documentation>
💡 Hint: Mention RunLLM in the thread for followups.
Grace C.
·
So for chain span kind, sometimes however it works that an Anthropic llm span is automatically instrumented - what could cause the behavior differences that sometimes it works and sometimes doesn't? Thanks!
Mikyo
·
Hey Grace C. any chance you have a minimum reproducible example? I’m actually not sure why this is happening
Grace C.
·
I don't have a reproducible script at the moment, unfortunately. However I do observe that for different span kind:
tool - always able to attach child anthropic auto-instrumented span
agent - sometimes unable to attach
chain - most of the time unable to attach
What's fundamentally different between them in terms of attaching children llm span?
Mikyo
·
Nothing really, that’s why I’m a bit confused. Are you using our decorators with an auto instrument or? Any insight into your setup would be very helpful
Grace C.
·
The agent span is a decorator, the anthropic llm is auto-instrumented
Mikyo
·
Hmm I’m still not quite sure what could be causing this. If you have any more details please let John G. and I know.
Grace C.
·
Hi Mikyo John G. I dig in more, and realize it has more to do with the anthropic llm auto-instrumentation - sometimes the llm span just isn't created, and it seems to be related to how fast the requests are being sent - whether it's triggered by human one after another slowly - all llm span can be captured, or programmatically one after another (which is apparently faster than human) - llm span can be dropped. We are using Anthropic streaming, and that's not supported in Phoenix at the moment, so we do the following:
stream = await self._anthropic_client.messages.create( max_tokens=self._max_tokens, system=self._system_prompt, model=self._model, messages=history.messages, thinking=thinking, tools=history.client_tools, stream=True, extra_headers=get_aigateway_extra_headers({"conversation-id": str(history.id)}), )
Our colleague Deepankar M. has created a feature request for anthropic stream method to be supported - https://github.com/Arize-ai/openinference/issues/1343 Would it be possible to fasten up this request to see if it solves the above issue? Thanks! 🙏
Mikyo
·
Ah thank you Grace C.! I'll work with the team to get this prioritized. Thank you for the details!

Understanding Span Kind Chain vs. Agent in LLM Instrumentation | Arize AI Community

RunLLM
APP
·
It seems likely that the difference in behavior between `chain` and `agent` span kinds could be due to how these spans are defined and used in Phoenix. According to the <https://docs.arize.com/phoenix/~/changes/v6Zhm276x8LlKmwqElIA/reference/open-inference#B%3A14|OpenInference Tracing documentation>, a `CHAIN` span is typically used for general logic operations or code blocks, while an `AGENT` span is used for invocations that encompass calls to LLMs and tools, often acting as a reasoning block. This distinction might affect how spans are automatically instrumented, especially if the `agent` span is designed to better capture the orchestration of LLM and tool interactions. Additionally, the <https://github.com/Arize-ai/phoenix/issues/1311|Phoenix GitHub issue> suggests that the `agent` span kind was introduced to delineate from `chain` because the attributes for this span kind are different. This could mean that the `agent` span is more suited for capturing complex interactions involving LLMs, which might explain why the `llm` span works better under `agent`. Would you like more detail on how to configure these spans or troubleshoot further? Sources: - <https://github.com/Arize-ai/phoenix/issues/1311|Phoenix GitHub issue> - <https://docs.arize.com/phoenix/~/changes/v6Zhm276x8LlKmwqElIA/reference/open-inference#B%3A14|OpenInference Tracing documentation>
💡 Hint: Mention RunLLM in the thread for followups.
Grace C.
·
So for chain span kind, sometimes however it works that an Anthropic llm span is automatically instrumented - what could cause the behavior differences that sometimes it works and sometimes doesn't? Thanks!
Mikyo
·
Hey Grace C. any chance you have a minimum reproducible example? I’m actually not sure why this is happening
Grace C.
·
I don't have a reproducible script at the moment, unfortunately. However I do observe that for different span kind:
tool - always able to attach child anthropic auto-instrumented span
agent - sometimes unable to attach
chain - most of the time unable to attach
What's fundamentally different between them in terms of attaching children llm span?
Mikyo
·
Nothing really, that’s why I’m a bit confused. Are you using our decorators with an auto instrument or? Any insight into your setup would be very helpful
Grace C.
·
The agent span is a decorator, the anthropic llm is auto-instrumented
Mikyo
·
Hmm I’m still not quite sure what could be causing this. If you have any more details please let John G. and I know.
Grace C.
·
Hi Mikyo John G. I dig in more, and realize it has more to do with the anthropic llm auto-instrumentation - sometimes the llm span just isn't created, and it seems to be related to how fast the requests are being sent - whether it's triggered by human one after another slowly - all llm span can be captured, or programmatically one after another (which is apparently faster than human) - llm span can be dropped. We are using Anthropic streaming, and that's not supported in Phoenix at the moment, so we do the following:
stream = await self._anthropic_client.messages.create( max_tokens=self._max_tokens, system=self._system_prompt, model=self._model, messages=history.messages, thinking=thinking, tools=history.client_tools, stream=True, extra_headers=get_aigateway_extra_headers({"conversation-id": str(history.id)}), )
Our colleague Deepankar M. has created a feature request for anthropic stream method to be supported - https://github.com/Arize-ai/openinference/issues/1343 Would it be possible to fasten up this request to see if it solves the above issue? Thanks! 🙏
Mikyo
·
Ah thank you Grace C.! I'll work with the team to get this prioritized. Thank you for the details!