Request to Add /v1/evaluations Endpoint to Klaviyo Project

Jordan C. · 2024-11-21T20:12:44.003Z

👋 Hey all, Jordan from Klaviyo here. I'm currently working on a project that will export evals manually (i.e. without the Phoenix SDK) - I noticed in your log_evaluations method (https://github.com/Arize-ai/phoenix/blob/27b872911a4a4083ece81218e2848f41d5edce14/src/phoenix/session/client.py#L290) you send a POST request to the /v1/evaluations endpoint, however I don't see that endpoint listed in your API (https://phoenix-demo.arize.com/apis). Would it be possible to add that or is there some reason it's excluded?

Request to Add /v1/evaluations Endpoint to Klaviyo Project | Arize AI Community

7 comments

· Sorted by Oldest

RunLLM
APP
·
The /v1/evaluations endpoint is used internally by the log_evaluations method in the Phoenix SDK to send evaluation data to the Phoenix server. However, it appears that this endpoint is not publicly documented in the API documentation available at the provided link. This could be due to the endpoint being intended for internal use or because the API documentation is not fully comprehensive.Would you like more details on how the log_evaluations method works or how you might manually interact with this endpoint? Sources: - Client.py in Phoenix GitHub
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
hi Jordan C. thanks so much for using Phoenix and our API! You're right, there are some routes we haven't fully published as part of our OpenAPI spec, let me discuss with the team whether or not we feel these evaluations endpoints are stable enough to publish and I'll get back to you soon
1
Dustin N.
·
hey Jordan C. we're going to add the routes to the schema! It should be released soon: https://github.com/Arize-ai/phoenix/pull/5505
Jordan C.
·
Amazing Dustin N., thanks for the quick turnaround. Much appreciated!
Jordan C.
·
Another request, sorry! We would like to scope this so we only need to use arize-phoenix-evals to run our eval suite and export them with Otel SDK. It doesn't seem like we can easily export them in the correct format without also installing the arize-phoenix package.log_evaluations() and some eval-related classes are only located in the arize-phoenix package and are inaccessible in the arize-phoenix-evals package. Is there a way we could forward these/move the source / types to the arize-phoenix-evals package so we don't need to add both to our dependencies?
Mikyo
·
Hey Jordan C. it's on our roadmap to de-couple the phoenix client from the main package. For now the best way to log evals without the package is simply through REST. the eval package is intended to be a de-coupled from logging. Hope that helps.
Jordan C.
·
Thanks for the reply Mikyo - I've been playing around with using http requests for the last couple days and making progress. Good to hear this is on your roadmap. I'll continue with using REST
🙌1

RunLLM
APP
·
The /v1/evaluations endpoint is used internally by the log_evaluations method in the Phoenix SDK to send evaluation data to the Phoenix server. However, it appears that this endpoint is not publicly documented in the API documentation available at the provided link. This could be due to the endpoint being intended for internal use or because the API documentation is not fully comprehensive.Would you like more details on how the log_evaluations method works or how you might manually interact with this endpoint? Sources: - Client.py in Phoenix GitHub
💡 Hint: Mention RunLLM in the thread for followups.
Dustin N.
·
hi Jordan C. thanks so much for using Phoenix and our API! You're right, there are some routes we haven't fully published as part of our OpenAPI spec, let me discuss with the team whether or not we feel these evaluations endpoints are stable enough to publish and I'll get back to you soon
1
Dustin N.
·
hey Jordan C. we're going to add the routes to the schema! It should be released soon: https://github.com/Arize-ai/phoenix/pull/5505
Jordan C.
·
Amazing Dustin N., thanks for the quick turnaround. Much appreciated!
Jordan C.
·
Another request, sorry! We would like to scope this so we only need to use arize-phoenix-evals to run our eval suite and export them with Otel SDK. It doesn't seem like we can easily export them in the correct format without also installing the arize-phoenix package.log_evaluations() and some eval-related classes are only located in the arize-phoenix package and are inaccessible in the arize-phoenix-evals package. Is there a way we could forward these/move the source / types to the arize-phoenix-evals package so we don't need to add both to our dependencies?
Mikyo
·
Hey Jordan C. it's on our roadmap to de-couple the phoenix client from the main package. For now the best way to log evals without the package is simply through REST. the eval package is intended to be a de-coupled from logging. Hope that helps.
Jordan C.
·
Thanks for the reply Mikyo - I've been playing around with using http requests for the last couple days and making progress. Good to hear this is on your roadmap. I'll continue with using REST
🙌1