Arize AI Community Icon

Home
Events

Arize AX Releases
Arize AX Support
Arize News
Arize Observe
Discussions
Introductions
Phoenix Support

Powered by Tightknit

Phoenix Support

Phoenix Support

LlamaIndex React vs OpenAI Agent: System Prompt Differences Explained

·Mar 29, 2024 09:59 PM

Anybody knows why when using Llamaindex React Agent we see system prompt + user + assistant panels in the AGENT span in phoenix But when using LlamaIndex OpenAI Agent we only see user + assistant (and tool) but not system prompt 🤔 First SC - React Agent Second SC - OpenAI Agent

14 comments

· Sorted by Oldest

Roger Y.
·
Hi Swaraj. Which llama-index module are you using? Are you using as_chat_engine with chat_mode="react" as shown here?
Swaraj R.
·
from llama_index.agent.openai import OpenAIAgent
Swaraj R.
·
Vs from llama_index.core.agent import ReactAgent
Swaraj R.
·
Their usage is pretty much the same We initialize using the “from_tools” function
Swaraj R.
·
We then make the call by using the arun_step method

·

for openai it looks like the system prompt needs to be entered manually. e.g.

from llama_index.core.agent.react.prompts import REACT_CHAT_SYSTEM_HEADER

agent = OpenAIAgent.from_tools(
    query_engine_tools,
    llm=llm,
    verbose=True,
    system_prompt=REACT_CHAT_SYSTEM_HEADER
)

Swaraj R.
·
hmm yeah we are not passing any custom system prompt that overrides the default function calling one I guess we shouldn't expect to see this because open ai agent likely using the function calling api? I was assuming phoenix would show an LLM span where the input includes all the tool schema descriptions (like in ReAct) But i suppose i'm having a wrong understanding , i guess that prompt would be on OpenAI side and likely not visible to Phoenix
Swaraj R.
·
I'll mark this as closed for now, and play around to see if anything comes up Thanks again
👍1
Roger Y.
·
i think the function schema in the actual openai request might not be captured in by the instrumentor. Alternatively, you can also instrument OpenAI itself with the following. You may have to put this at the very top of your program so it runs before importing llama-index
from phoenix.trace.openai import OpenAIInstrumentor OpenAIInstrumentor().instrument()
Roger Y.
·
^ this will capture the actual JSON object sent to OpenAI and it should contain the function schema
Swaraj R.
·
oh nice okay let me try this !
Swaraj R.
·
thank you!
Roger Y.
·
np
Roger Y.
·
you may have to restart your program if llama-index is already imported into memory
🙌1