Hi, would like to ask about the span kind chain vs. agent - sometimes I realize the anthropic llm span isn't automatically instrumented for under chain but works fine with agent - what could be the reason? Thanks! 馃檹
馃挕聽Hint: Mention RunLLM in the thread for followups.
So for chain span kind, sometimes however it works that an Anthropic llm span is automatically instrumented - what could cause the behavior differences that sometimes it works and sometimes doesn't? Thanks!
I don't have a reproducible script at the moment, unfortunately. However I do observe that for different span kind:
tool - always able to attach child anthropic auto-instrumented span
agent - sometimes unable to attach
chain - most of the time unable to attach
What's fundamentally different between them in terms of attaching children llm span?
Nothing really, that鈥檚 why I鈥檓 a bit confused. Are you using our decorators with an auto instrument or? Any insight into your setup would be very helpful
The agent span is a decorator, the anthropic llm is auto-instrumented
Hi Mikyo John G. I dig in more, and realize it has more to do with the anthropic llm auto-instrumentation - sometimes the llm span just isn't created, and it seems to be related to how fast the requests are being sent - whether it's triggered by human one after another slowly - all llm span can be captured, or programmatically one after another (which is apparently faster than human) - llm span can be dropped. We are using Anthropic streaming, and that's not supported in Phoenix at the moment, so we do the following:
stream = await self._anthropic_client.messages.create(
max_tokens=self._max_tokens,
system=self._system_prompt,
model=self._model,
messages=history.messages,
thinking=thinking,
tools=history.client_tools,
stream=True,
extra_headers=get_aigateway_extra_headers({"conversation-id": str(history.id)}),
)Our colleague Deepankar M. has created a feature request for anthropic stream method to be supported - https://github.com/Arize-ai/openinference/issues/1343 Would it be possible to fasten up this request to see if it solves the above issue? Thanks! 馃檹