Questions on Arize Phoenix Instrumentation with CrewAI Agent

Xander S. · 2024-10-25T07:32:33.561Z

@Arun Reposting from #ml-observability here (https://arize-ai.slack.com/archives/C035FRHT9G8/p1729839360763859) Hi, I am starting to use Arize pheonize instrumentation with my CrewAI agent. I have few questions: 1. How can I get the tools called and the tool outputs from the traces? 2. I am currently using crewai==0.11.0 and from this doc (https://docs.arize.com/phoenix/tracing/integrations-tracing/crewai), I am using Langchain instrumentor to get the traces. Should I also use crewaiinstrumentor as well? I am not sure whats the difference between both. 3. How can I get the trace for multiple data points separately in a project? My crewai agent contains multiple internal agents working together. I expected ideally for a single data point, there is one row created by Pheonix, but I see for a single data point, multiple spans are created. If I run multiple data points with the same project, there is no demarcation between spans for different data points. How to do this? I basically want to get the traces for data point separately. 4. Also, I am using OpenAI models beneath the agents and can I assume total_tokens output in the project is the actual value? or is it possible some of calls may not be logged? Thanks.

Xander S.

·Oct 25, 2024 07:32 AM

Arun Reposting from 🔒[private feed] here

Hi, I am starting to use Arize pheonize instrumentation with my CrewAI agent. I have few questions:

1.
How can I get the tools called and the tool outputs from the traces?
2.
I am currently using crewai==0.11.0 and from this doc (https://docs.arize.com/phoenix/tracing/integrations-tracing/crewai), I am using Langchain instrumentor to get the traces. Should I also use crewaiinstrumentor as well? I am not sure whats the difference between both.
3.
How can I get the trace for multiple data points separately in a project? My crewai agent contains multiple internal agents working together. I expected ideally for a single data point, there is one row created by Pheonix, but I see for a single data point, multiple spans are created. If I run multiple data points with the same project, there is no demarcation between spans for different data points. How to do this? I basically want to get the traces for data point separately.
4.
Also, I am using OpenAI models beneath the agents and can I assume total_tokens output in the project is the actual value? or is it possible some of calls may not be logged?

Thanks.

8 comments