Hi, my requirement is to extract llm evaluation outputs of scores and explanation (both in built and custom) from Arize platform inorder to display that in an external dashboard. I have been trying graph ql api. Appears there is no direct way to access metrics but was supposed to be possible using monitors or dashboards created for the llm evaluation score. Havent found any success so far. Can someone help me with the graph ql which can fetch the date, score and explanation for the llm evals on a project? I also tried SDK, but it gives the "flight unavailable error" on exporter client.export_model_to_df. Please help
I would like to raise the support ticket for resolution on this issue. I either need a graph ql api using monitors or dashboard to retrieve llm eval metrics and metrics like cost,latency. Or I need an SDK to carry out this extraction.
hey Rajeswari K., I'm also trying to export the metrics and was wondering if you have resolved this issue
Hey 🔒[private user] I've tried this "import json import os from phoenix.client import Client from dotenv import load_dotenv def get_all_spans(): """Fetch all spans from Arize Phoenix using the official client.""" # Initialize Phoenix client client = Client( api_key=PHOENIX_API_KEY, base_url=BASE_URL, headers={"api_key": PHOENIX_API_KEY}, ) # Fetch spans using the official client spans = client.spans.get_spans(project_identifier=PROJECT_ID) return spans"
🔒[private user], sure I'll try that. Will it also output the latency information? And I am on the new cloud Phoenix version
🔒[private user] yup this is the error I'm getting
PS C:\Users\T767303\Desktop\Rapids Repos\phoenix> & C:/Users/T767303/AppData/Local/Programs/Python/Python313/python.exe "c:/Users/T767303/Desktop/Rapids Repos/phoenix/import_phoenix.py"
Arize Phoenix is not running on https://app.phoenix.arize.com/s/rapidshealth/. Launch Phoenix with `import phoenix as px; px.launch_app()`
Traceback (most recent call last):
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 101, in map_httpcore_exceptions
yield
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 256, in handle_request
raise exc from None
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 236, in handle_request
response = connection.handle_request(
pool_request.request
)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\http_proxy.py", line 288, in handle_request
connect_response = self._connection.handle_request(
connect_request
)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 101, in handle_request
raise exc
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 78, in handle_request
stream = self._connect(request)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_backends\sync.py", line 207, in connect_tcp
with map_exceptions(exc_map):
~~~~~~~~~~~~~~^^^^^^^^^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 156, in query_spans
response = self._client.post(
url="v1/spans",
...<13 lines>...
timeout=timeout,
)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1144, in post
return self.request(
~~~~~~~~~~~~^
"POST",
^^^^^^^
...<11 lines>...
extensions=extensions,
^^^^^^^^^^^^^^^^^^^^^^
)
^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\utilities\client.py", line 76, in request
response = super().request(*args, **kwargs)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 825, in request
return self.send(request, auth=auth, follow_redirects=follow_redirects)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 914, in send
response = self._send_handling_auth(
request,
...<2 lines>...
history=[],
)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
request,
follow_redirects=follow_redirects,
history=history,
)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 249, in handle_request
with map_httpcore_exceptions():
~~~~~~~~~~~~~~~~~~~~~~~^^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
self.gen.throw(value)
~~~~~~~~~~~~~~^^^^^^^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 38, in <module>
main()
~~~~^^
File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 26, in main
spans = get_all_spans()
File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 19, in get_all_spans
phoenix_df = px.Client().get_spans_dataframe(project_name='main | transformation', timeout=None)
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\data_extractor.py", line 46, in get_spans_dataframe
self.query_spans(
~~~~~~~~~~~~~~~~^
SpanQuery().where(filter_condition or ""),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<5 lines>...
timeout=timeout,
^^^^^^^^^^^^^^^^
),
^
File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 187, in query_spans
raise TimeoutError(error_message) from error
phoenix.session.client.TimeoutError: The request timed out. The timeout can be adjusted by passing a number of seconds to the `timeout` parameter and can be disabled altogether by passing `None`.
I've provided the api key in the header and the collector endpoint which is "https://app.phoenix.arize.com/s/<instancename>
Arize Phoenix is not running on https://app.phoenix.arize.com/s/... Launch Phoenix with `import phoenix as px; px.launch_app()` Traceback (most recent call last): File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 27, in <module> main() ~~~~^^ File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 20, in main phoenix_df = px.Client().get_spans_dataframe(project_identifier="fgjgjlljklk") TypeError: TraceDataExtractor.get_spans_dataframe() got an unexpected keyword argument 'project_identifier' 🔒[private user]
I'm not sure, how can i ensure that?
this takes me to a different page. I dont see my projects if i log in through that
I use the same website and when I log in using the welcome email link i recieved I'm able to see my projects. I just noticed this link also has phoenix cloud on it. Not sure why the link u sent me takes me to antother link with none of my projects even though I logged in using the same account
I actually got a sign up link from my team which created this and added all the projects in
the link has /s/ in it