Hi is there a prompt management feature that also connects to the code version and can execute that code from the prompt playground (the whole trace) instead of just instrumenting the llm calls? Is that planned for AX? here is an example of what I mean: https://langfuse.com/docs/evaluation/dataset-runs/native-run Also the agent graph is ok, but a framework-agnostic "LangGraph Studio" would be much more powerful. Thanks
not at all, if you read the link I sent it’s very different…arize only applies the prompt to the dataset, the feature I mentioned can run the whole trace remotely using the original code
so other tool calls or business logic would also run
Tiago F. Answering for the product team here. I believe the feature you dropped is designed to run a prompt over a dataset as an online task ("native"). There are features we are working on that support external calls to Evals (so you can run them on your side) or Agent replay (also options to run in your infrastructure). But the feature above you dropped the link to is neither of those. Do you want the Agent to run in your code, kicked off by an online task from Arize or playground in our system? Or are you running the task and just logging the data? If the former you need to define APIs and callbacks between the systems. Feel free to point us to Langfuse examples that show the callbacks and we can dig deeper. Or just happy to brainstorm what you need and talk about what we have upcoming.
