Improving Arize for Open and Axial Coding Efficiency
I am struggling to make Arize efficient to perform open coding and axial coding on traces (to evaluate the overall performance of the LLM project):
- 1.
Annotations only allow labels (e.g. good, bad , ..etc) or scores (min score, max score). It would be nice to be able to use free text to note general qualitative comments about a trace so that this information is fed to an LLM judge for further classification and error analysis without having to use an external tool (e.g. CSV files).
- 2.
I tried to leverage the notes field but alas, the filters on the UI do not allow me to filter by this field and also the exports do not include this field 馃槩
- 3.
When I add/remove columns to the project view (e.g. specific span attributes) and I try to export the spans to a CSV file, some of the columns that are visible on the UI are missing from the CSV file. It would be helpful if the download feature respected the filters and the column customizations applied to the view itself.