Our friends over at Zenbase (DSPy contributors) are working on prompt optimization based on human feedback. Should work really nicely with a lot of the annotation improvements coming to Arize Phoenix. Reach out to them if you are interested!
https://www.notion.so/zenbase-ai/Automating-User-Feedback-to-Prompt-Improvements-1e0022301ecb80f98795e62333030b02?pvs=4