Damith S.

Commented on How to Automate Session Selection and Upload Conve...·Posted inArize AX Support

Thanks 🔒[private user]. Currently going with the Python sdk + experiment creaction via TS as you recommended. Keen to know what other features are included in the new roll outs. We're keen to see if there is a more comprehensive Typescript support suite which closes the gap on python 🙂

Posted in Arize AX Support·

Damith S.

How to Automate Session Selection and Upload Conversations to Arize?

Hi Guys, I wan to set up some scripts to facilitate my evals:

1.
select a set of sessions in a history window (last 1 day, 7 days etc). Project/Model of my choice.
2.
Parse out the conversations and upload as a dataset to Arize.

Ideally this needs to be done in typescript but open for suggestions using Python as well. What is the best way to do this ? I can't seem to find an API/ Integration to do this programmatically.

1Comment

Commented on How to Upload Experiment Runs and Results to AX Us...·Posted inArize AX Support

Damith S.

Thanks Srilakshmi C.. Where can I find the reference on how to upload the LLM as a judge results into the experiment run? I'm referring to this and it's given info on how to upload the output but not the evaluator results. https://arize.com/docs/api-reference/experiments/create-an-experiment

Posted in Arize AX Support·

Damith S.

How to Upload Experiment Runs and Results to AX Using Typescript?

hi guys, any way for me to find out how to upload experiment runs to AX ? my plan is to retrieve a dataset from AX, run a task + evals for each example, and run the output + eval results back to AX (like we do with Phoenix). Using typescript, I can see the api ref has docs to create experiments but not write experiment results. Appreciate your help 🙏

3Comments

Commented on Setting Up Longitudinal Framework for Eval Perform...·Posted inPhoenix Support

Damith S.

John G., thanks for this! It's confirming a pattern we've been thinking of adopting (cdn to store audio). Cheers.

Commented on Setting Up Longitudinal Framework for Eval Perform...·Posted inPhoenix Support

Damith S.

Would be a really big plus to have

Commented on Setting Up Longitudinal Framework for Eval Perform...·Posted inPhoenix Support

Damith S.

Thanks Mikyo (sorry for radio silence). Also, wondering if anyone has used Phoenix for voice evals. E.g., for openai's realtime api evaluations.

Commented on Setting Up Longitudinal Framework for Eval Perform...·Posted inPhoenix Support

Damith S.

Hi Jason, basically I want to set up an evaluator for an experiment, and see a chart of how this evaluator performed over multiple experiment runs. Currently, I can see them listed in a table but is there a feature to view them as a time-series over time?

Posted in Phoenix Support·

Damith S.

Setting Up Longitudinal Framework for Eval Performance in Phoenix

Hi guys, having a lot of phun ( 😉 ) using phoenix in our dev workflow.. We've been attempting to set up a longitudinal framework which shows us performance of evals across multiple experiments. In the self-hosted docker version this doesn't seem to be supported. Is this a premium feature?

8Comments