Hi, would like to ask about the span kind chain vs. agent - sometimes I realize the anthropic llm span isn't automatically instrumented for under chain but works fine with agent - what could be the reason? Thanks! 馃檹
It seems likely that the difference in behavior between chain and agent span kinds could be due to how these spans are defined and used in Phoenix. According to the OpenInference Tracing documentation, a CHAIN span is typically used for general logic operations or code blocks, while an AGENT span is used for invocations that encompass calls to LLMs and tools, often acting as a reasoning block. This distinction might affect how spans are automatically instrumented, especially if the agent span is designed to better capture the orchestration of LLM and tool interactions.Additionally, the Phoenix GitHub issue suggests that the agent span kind was introduced to delineate from chain because the attributes for this span kind are different. This could mean that the agent span is more suited for capturing complex interactions involving LLMs, which might explain why the llm span works better under agent.Would you like more detail on how to configure these spans or troubleshoot further? Sources:
- Phoenix GitHub issue
- OpenInference Tracing documentation
馃挕聽Hint: Mention RunLLM in the thread for followups.
So for chain span kind, sometimes however it works that an Anthropic llm span is automatically instrumented - what could cause the behavior differences that sometimes it works and sometimes doesn't? Thanks!
I don't have a reproducible script at the moment, unfortunately. However I do observe that for different span kind:
tool - always able to attach child anthropic auto-instrumented span
agent - sometimes unable to attach
chain - most of the time unable to attach
What's fundamentally different between them in terms of attaching children llm span?
Nothing really, that鈥檚 why I鈥檓 a bit confused. Are you using our decorators with an auto instrument or? Any insight into your setup would be very helpful
The agent span is a decorator, the anthropic llm is auto-instrumented
Hi Mikyo John G. I dig in more, and realize it has more to do with the anthropic llm auto-instrumentation - sometimes the llm span just isn't created, and it seems to be related to how fast the requests are being sent - whether it's triggered by human one after another slowly - all llm span can be captured, or programmatically one after another (which is apparently faster than human) - llm span can be dropped. We are using Anthropic streaming, and that's not supported in Phoenix at the moment, so we do the following:
stream = await self._anthropic_client.messages.create(
max_tokens=self._max_tokens,
system=self._system_prompt,
model=self._model,
messages=history.messages,
thinking=thinking,
tools=history.client_tools,
stream=True,
extra_headers=get_aigateway_extra_headers({"conversation-id": str(history.id)}),
)Our colleague Deepankar M. has created a feature request for anthropic stream method to be supported - https://github.com/Arize-ai/openinference/issues/1343 Would it be possible to fasten up this request to see if it solves the above issue? Thanks! 馃檹