Hi, I'm encountering an issue with tool definitions. In my tool, some parameter keys are dependent on others, as referenced in their descriptions鈥攕o the order of keys is important. However, Phoenix stores tool definitions in a JSONB column, which automatically reorders the keys (typically by key length), and this is causing the tool to malfunction. Is there a way to preserve the key order or work around this issue? Also, would it be possible to use a JSON column instead of JSONB to maintain the original key order?
The Phoenix's documentation I have ingested does not mention how to preserve the key order in JSONB columns or whether using a JSON column instead would maintain the original key order. You might want to provide additional context on how the tool definitions are being used or any specific database operations that are affected by the key order. If this doesn't help, please reach out to the Phoenix support team for further assistance.
馃挕聽Hint: Mention RunLLM in the thread for followups.
Sorry for the duplicate response, already responded in the other thread. Will paste here as well
Hey there, thanks for reaching out and sorry you are seeing issues. I would not expect key order to impact behavior, that's interesting. Can you describe a bit more about your use case where key order impacts performance? The json specification does not take a strong stance on object key order but it does recommend that object keys are treated as unordered for maximum compatibility. I think I would agree with that stance as every touch point that the tool json passes through may or may not preserve key order, and I'm not sure that we'd like to limit our technical design in this way. I am curious what others think though.
No worries at all, I actually missed this one 馃槄
Thanks for your response, Anthony P.. In my tool call definition, I referenced one parameter within the description of another, using it as a condition. According to several sources, OpenAI processes parameters from top to bottom. If a key is referenced before it's defined, it can lead to hallucinations and incorrect responses. I tested this by rearranging the parameter order, and when defined correctly, it worked as expected. In my case, the value of certain keys should depend on another key鈥攅ither being null or an object鈥攂ut it always returns null regardless.
Are you seeing the key reordering behavior when you save your tool definition as a prompt? Phoenix does not store tool definitions otherwise (besides in traces)
Yes, I am storing tools definition with prompt and fetching them using API.
Got it. And one more clarification, are you storing tool calls in your prompt as well, or just tool definitions
I am storing tool calls in Tools as shown in the below image.
I have created the following issue and will bring it up with the team, thanks for bringing this up 馃檹 https://github.com/Arize-ai/phoenix/issues/7254
Feel free to add any additional details, or examples you wish to the issue
Thanks for logging it! 馃檹 Anthony P.
In the meantime, I am curious if restructuring your tool definition to use an array of named objects would suit your use case, as arrays are ordered by nature.
OpenAI's tool call definition doesn't support arrays鈥攊t requires the use of JSON objects.
The top level needs to be an object, but the properties themselves do not
