Deepanshu

Claude 4 models are meaningfully better on code benchmarks than o3 — the best reasoning model so far. And Anthropic is bundling web search, a Python execution sandbox and a files API 🤯 As far as dev-ex for agents goes, I think Anthropic has pulled ahead now!

Hello, What’s the best way to automatically track prompt changes (committed via Git) and evaluate their outputs against predefined ground truths? The idea is to have a workflow that gets triggered whenever a developer updates a prompt in Git. This workflow should run evaluations comparing the new prompt outputs against a set of ground truth responses to ensure consistency or detect regressions.

Does Arize supports clustering/analysis of all the user's questions/prompts and help visualise them?

Is there any model router implementation that folks use for determining which LLM to use based on user's query?

Want to implement a good summarisation engine for million of documents, which non-LLM model would be the best to create document summary?