Streamlining Prompt Versions and Langgraph Message Formatting

Rasmus B. · 2025-12-04 01:48:55.275602+00

I might be going against the grain here, but is there a reason that model_name is a forced parameter when defining prompt versions? For sure different models can require variant prompts, but for some things I just want to manage a single version of the prompt. I am also using Langgraph, and it's a bit annoying to have to first format to provider messages and then convert to Langgraph/Langchain messages? Is there a more streamlined approach to get either raw prompt text/templates or messages for a particular SDK?

Streamlining Prompt Versions and Langgraph Message Formatting | Arize AI Community

9 comments

· Sorted by Oldest

RunLLM
APP
·
·
Phoenix requires specifying model_name when defining prompt versions to ensure reproducibility and compatibility, as different models may interpret prompts differently or require unique formatting. This design is intentional to support versioning, tracking, and provider-specific prompt adaptation, not just for OpenAI but for Anthropic, Google, and others as well—see promptVersion docs and specs.For SDK integration, Phoenix provides helpers like toSDK to convert prompts to the format expected by OpenAI, Anthropic, or Vercel AI SDKs, but there is currently no built-in method to directly output prompts in Langchain/Langgraph format; users must manually convert or extract the raw template/messages and adapt as needed (docs, enhancement request). Would you like more detail on extracting raw templates or automating conversion for Langchain?Sources: - using-a-prompt - promptVersion docs - specs - enhancement request
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
·
hi Rasmus B. that's totally understandable, but we specify model names and providers for each prompt because in general invocation parameters etc are not identical from model to model, and one of the hopes for the "prompt" concept was we could use it to reproduce the conditions you used to get a response with as much fidelity as possible
Roger Y.
·
·
What Dustin said is correct, and that’s mostly from the server’s point of view — the provider aspect helps track prompts that need that kind of differentiation. From the client’s perspective, after the prompt is retrieved from the server, the model name and provider are mostly ancillary and “separable” from the prompt (they just provide a starting point, and the prompts themselves are mostly neutral). I think what we’re really missing are just helper functions to translate the prompt into the LangChain format, similar to what we’ve done for the OpenAI format.
Rasmus B.
·
·
Dustin N. Roger Y. I can totally see what you are going for, but feels a bit strict to require attachment of specific metadata to the prompt even if the user doesn't need it? If extended to work with frameworks like Langchain, is the idea that you'd set model_provider="LANGCHAIN" or something like that? For what it's worth, I found today that langchain_core.messages.utils.convert_to_messages converts OpenAI message formats pretty painlessly, but it still seems like an odd roundtrip to first convert it to OpenAI format.
Roger Y.
·
·
Could you share a brief snippet or sketch of your current—or ideal—prompt workflow? (DM is fine if you prefer.) Seeing a concrete example would help us pinpoint where we can make things smoother for your use case.
Rasmus B.
·
·
Roger Y. I'm honestly still figuring it out, but we are taking a modular approach to prompts (so prompts are composed of multiple versioned sub-prompts) and in addition to pure instructional prompts we have context prompts, e.g. we have a description of our core data model to help with using query tools. This data model then gets embedded into multiple system prompts to provide context for different operations in the agent graph. So since I also need to do post-hoc composition, in some sense it would be easier for me if I could just version pure strings/templates, whereas now I have to do string -> system message-> upload -> phoenix store -> download -> versioned prompt -> openai message -> text -> compose. To make better use of the message storage format I could maybe version the composed prompts separately, in which case I could plug it directly into Langgraph (by converting to the format), but there is some overhead with tracking how the composed prompt is pieced together from particular sub-prompt versions. Don't know if modular prompts are on the roadmap in some way or completely out of scope 🙂
Roger Y.
·
·
Thanks for the detailed context! The modular prompt composition pattern is an interesting extension, so it’s really helpful to understand your pain points in practice. I can see how the current setup can add friction when you’re trying to assemble prompts from versioned components, and yeah, tracking which sub-prompt versions went into a composed prompt can be a challenge in general. This is definitely something worth exploring. It’s not something we can prioritize immediately, but it’s on our radar now. If you find cleaner patterns or hit major blockers in the meantime, please let us know!
Rasmus B.
·
·
Roger Y. looked around a bit for what others were doing, and there isn't a lot of support for modular prompts around to be fair, but I did notice that Langfuse has some functionality for it: https://langfuse.com/docs/prompt-management/features/composability
Roger Y.
·
·
Thanks for sharing. That’s good inspiration as we think about how modular prompts could evolve down the line.

RunLLM
APP
·
·
Phoenix requires specifying model_name when defining prompt versions to ensure reproducibility and compatibility, as different models may interpret prompts differently or require unique formatting. This design is intentional to support versioning, tracking, and provider-specific prompt adaptation, not just for OpenAI but for Anthropic, Google, and others as well—see promptVersion docs and specs.For SDK integration, Phoenix provides helpers like toSDK to convert prompts to the format expected by OpenAI, Anthropic, or Vercel AI SDKs, but there is currently no built-in method to directly output prompts in Langchain/Langgraph format; users must manually convert or extract the raw template/messages and adapt as needed (docs, enhancement request). Would you like more detail on extracting raw templates or automating conversion for Langchain?Sources: - using-a-prompt - promptVersion docs - specs - enhancement request
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
·
hi Rasmus B. that's totally understandable, but we specify model names and providers for each prompt because in general invocation parameters etc are not identical from model to model, and one of the hopes for the "prompt" concept was we could use it to reproduce the conditions you used to get a response with as much fidelity as possible
Roger Y.
·
·
What Dustin said is correct, and that’s mostly from the server’s point of view — the provider aspect helps track prompts that need that kind of differentiation. From the client’s perspective, after the prompt is retrieved from the server, the model name and provider are mostly ancillary and “separable” from the prompt (they just provide a starting point, and the prompts themselves are mostly neutral). I think what we’re really missing are just helper functions to translate the prompt into the LangChain format, similar to what we’ve done for the OpenAI format.
Rasmus B.
·
·
Dustin N. Roger Y. I can totally see what you are going for, but feels a bit strict to require attachment of specific metadata to the prompt even if the user doesn't need it? If extended to work with frameworks like Langchain, is the idea that you'd set model_provider="LANGCHAIN" or something like that? For what it's worth, I found today that langchain_core.messages.utils.convert_to_messages converts OpenAI message formats pretty painlessly, but it still seems like an odd roundtrip to first convert it to OpenAI format.
Roger Y.
·
·
Could you share a brief snippet or sketch of your current—or ideal—prompt workflow? (DM is fine if you prefer.) Seeing a concrete example would help us pinpoint where we can make things smoother for your use case.
Rasmus B.
·
·
Roger Y. I'm honestly still figuring it out, but we are taking a modular approach to prompts (so prompts are composed of multiple versioned sub-prompts) and in addition to pure instructional prompts we have context prompts, e.g. we have a description of our core data model to help with using query tools. This data model then gets embedded into multiple system prompts to provide context for different operations in the agent graph. So since I also need to do post-hoc composition, in some sense it would be easier for me if I could just version pure strings/templates, whereas now I have to do string -> system message-> upload -> phoenix store -> download -> versioned prompt -> openai message -> text -> compose. To make better use of the message storage format I could maybe version the composed prompts separately, in which case I could plug it directly into Langgraph (by converting to the format), but there is some overhead with tracking how the composed prompt is pieced together from particular sub-prompt versions. Don't know if modular prompts are on the roadmap in some way or completely out of scope 🙂
Roger Y.
·
·
Thanks for the detailed context! The modular prompt composition pattern is an interesting extension, so it’s really helpful to understand your pain points in practice. I can see how the current setup can add friction when you’re trying to assemble prompts from versioned components, and yeah, tracking which sub-prompt versions went into a composed prompt can be a challenge in general. This is definitely something worth exploring. It’s not something we can prioritize immediately, but it’s on our radar now. If you find cleaner patterns or hit major blockers in the meantime, please let us know!
Rasmus B.
·
·
Roger Y. looked around a bit for what others were doing, and there isn't a lot of support for modular prompts around to be fair, but I did notice that Langfuse has some functionality for it: https://langfuse.com/docs/prompt-management/features/composability
Roger Y.
·
·
Thanks for sharing. That’s good inspiration as we think about how modular prompts could evolve down the line.