Extracting LLM Evaluation Scores from Arize Platform API

·Jul 09, 2025 07:17 AM

Hi, my requirement is to extract llm evaluation outputs of scores and explanation (both in built and custom) from Arize platform inorder to display that in an external dashboard. I have been trying graph ql api. Appears there is no direct way to access metrics but was supposed to be possible using monitors or dashboards created for the llm evaluation score. Havent found any success so far. Can someone help me with the graph ql which can fetch the date, score and explanation for the llm evals on a project? I also tried SDK, but it gives the "flight unavailable error" on exporter client.export_model_to_df. Please help

13 comments

· Sorted by Oldest

Rajeswari K.
·
I would like to raise the support ticket for resolution on this issue. I either need a graph ql api using monitors or dashboard to retrieve llm eval metrics and metrics like cost,latency. Or I need an SDK to carry out this extraction.
👀1
➕1
Suhrullekha C.
·
hey Rajeswari K., I'm also trying to export the metrics and was wondering if you have resolved this issue
Suhrullekha C.
·
Hey 🔒[private user] I've tried this "import json import os from phoenix.client import Client from dotenv import load_dotenv def get_all_spans(): """Fetch all spans from Arize Phoenix using the official client.""" # Initialize Phoenix client client = Client( api_key=PHOENIX_API_KEY, base_url=BASE_URL, headers={"api_key": PHOENIX_API_KEY}, ) # Fetch spans using the official client spans = client.spans.get_spans(project_identifier=PROJECT_ID) return spans"
Suhrullekha C.
·
🔒[private user], sure I'll try that. Will it also output the latency information? And I am on the new cloud Phoenix version
Suhrullekha C.
·
🔒[private user] yup this is the error I'm getting

Suhrullekha C.

PS C:\Users\T767303\Desktop\Rapids Repos\phoenix> & C:/Users/T767303/AppData/Local/Programs/Python/Python313/python.exe "c:/Users/T767303/Desktop/Rapids Repos/phoenix/import_phoenix.py"
Arize Phoenix is not running on https://app.phoenix.arize.com/s/rapidshealth/. Launch Phoenix with `import phoenix as px; px.launch_app()`
Traceback (most recent call last):
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 101, in map_httpcore_exceptions
    yield
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 250, in handle_request
    resp = self._pool.handle_request(req)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 256, in handle_request
    raise exc from None
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 236, in handle_request
    response = connection.handle_request(
        pool_request.request
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\http_proxy.py", line 288, in handle_request
    connect_response = self._connection.handle_request(
        connect_request
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 101, in handle_request
    raise exc
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 78, in handle_request
    stream = self._connect(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 124, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_backends\sync.py", line 207, in connect_tcp
    with map_exceptions(exc_map):
         ~~~~~~~~~~~~~~^^^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 156, in query_spans
    response = self._client.post(
        url="v1/spans",
    ...<13 lines>...
        timeout=timeout,
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1144, in post
    return self.request(
           ~~~~~~~~~~~~^
        "POST",
        ^^^^^^^
    ...<11 lines>...
        extensions=extensions,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\utilities\client.py", line 76, in request
    response = super().request(*args, **kwargs)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 914, in send
    response = self._send_handling_auth(
        request,
    ...<2 lines>...
        history=[],
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
        request,
        follow_redirects=follow_redirects,
        history=history,
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 249, in handle_request
    with map_httpcore_exceptions():
         ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 38, in <module>
    main()
    ~~~~^^
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 26, in main
    spans = get_all_spans()
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 19, in get_all_spans
    phoenix_df = px.Client().get_spans_dataframe(project_name='main | transformation', timeout=None)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\data_extractor.py", line 46, in get_spans_dataframe
    self.query_spans(
    ~~~~~~~~~~~~~~~~^
        SpanQuery().where(filter_condition or ""),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
        timeout=timeout,
        ^^^^^^^^^^^^^^^^
    ),
    ^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 187, in query_spans
    raise TimeoutError(error_message) from error
phoenix.session.client.TimeoutError: The request timed out. The timeout can be adjusted by passing a number of seconds to the `timeout` parameter and can be disabled altogether by passing `None`.

Suhrullekha C.
·
I've provided the api key in the header and the collector endpoint which is "https://app.phoenix.arize.com/s/<instancename>
Suhrullekha C.
·
Arize Phoenix is not running on https://app.phoenix.arize.com/s/... Launch Phoenix with `import phoenix as px; px.launch_app()` Traceback (most recent call last): File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 27, in <module> main() ~~~~^^ File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 20, in main phoenix_df = px.Client().get_spans_dataframe(project_identifier="fgjgjlljklk") TypeError: TraceDataExtractor.get_spans_dataframe() got an unexpected keyword argument 'project_identifier' 🔒[private user]
Suhrullekha C.
·
I'm not sure, how can i ensure that?
Suhrullekha C.
·
this takes me to a different page. I dont see my projects if i log in through that
Suhrullekha C.
·
I use the same website and when I log in using the welcome email link i recieved I'm able to see my projects. I just noticed this link also has phoenix cloud on it. Not sure why the link u sent me takes me to antother link with none of my projects even though I logged in using the same account
Suhrullekha C.
·
I actually got a sign up link from my team which created this and added all the projects in
Suhrullekha C.
·
the link has /s/ in it

Rajeswari K.
·
I would like to raise the support ticket for resolution on this issue. I either need a graph ql api using monitors or dashboard to retrieve llm eval metrics and metrics like cost,latency. Or I need an SDK to carry out this extraction.
👀1
➕1
Suhrullekha C.
·
hey Rajeswari K., I'm also trying to export the metrics and was wondering if you have resolved this issue
Suhrullekha C.
·
Hey 🔒[private user] I've tried this "import json import os from phoenix.client import Client from dotenv import load_dotenv def get_all_spans(): """Fetch all spans from Arize Phoenix using the official client.""" # Initialize Phoenix client client = Client( api_key=PHOENIX_API_KEY, base_url=BASE_URL, headers={"api_key": PHOENIX_API_KEY}, ) # Fetch spans using the official client spans = client.spans.get_spans(project_identifier=PROJECT_ID) return spans"
Suhrullekha C.
·
🔒[private user], sure I'll try that. Will it also output the latency information? And I am on the new cloud Phoenix version
Suhrullekha C.
·
🔒[private user] yup this is the error I'm getting

Suhrullekha C.

PS C:\Users\T767303\Desktop\Rapids Repos\phoenix> & C:/Users/T767303/AppData/Local/Programs/Python/Python313/python.exe "c:/Users/T767303/Desktop/Rapids Repos/phoenix/import_phoenix.py"
Arize Phoenix is not running on https://app.phoenix.arize.com/s/rapidshealth/. Launch Phoenix with `import phoenix as px; px.launch_app()`
Traceback (most recent call last):
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 101, in map_httpcore_exceptions
    yield
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 250, in handle_request
    resp = self._pool.handle_request(req)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 256, in handle_request
    raise exc from None
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection_pool.py", line 236, in handle_request
    response = connection.handle_request(
        pool_request.request
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\http_proxy.py", line 288, in handle_request
    connect_response = self._connection.handle_request(
        connect_request
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 101, in handle_request
    raise exc
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 78, in handle_request
    stream = self._connect(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_sync\connection.py", line 124, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_backends\sync.py", line 207, in connect_tcp
    with map_exceptions(exc_map):
         ~~~~~~~~~~~~~~^^^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 156, in query_spans
    response = self._client.post(
        url="v1/spans",
    ...<13 lines>...
        timeout=timeout,
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1144, in post
    return self.request(
           ~~~~~~~~~~~~^
        "POST",
        ^^^^^^^
    ...<11 lines>...
        extensions=extensions,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\utilities\client.py", line 76, in request
    response = super().request(*args, **kwargs)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 914, in send
    response = self._send_handling_auth(
        request,
    ...<2 lines>...
        history=[],
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
        request,
        follow_redirects=follow_redirects,
        history=history,
    )
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_client.py", line 1014, in _send_single_request
    response = transport.handle_request(request)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 249, in handle_request
    with map_httpcore_exceptions():
         ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\httpx\_transports\default.py", line 118, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 38, in <module>
    main()
    ~~~~^^
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 26, in main
    spans = get_all_spans()
  File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 19, in get_all_spans
    phoenix_df = px.Client().get_spans_dataframe(project_name='main | transformation', timeout=None)
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\data_extractor.py", line 46, in get_spans_dataframe
    self.query_spans(
    ~~~~~~~~~~~~~~~~^
        SpanQuery().where(filter_condition or ""),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
        timeout=timeout,
        ^^^^^^^^^^^^^^^^
    ),
    ^
  File "C:\Users\T767303\AppData\Local\Programs\Python\Python313\Lib\site-packages\phoenix\session\client.py", line 187, in query_spans
    raise TimeoutError(error_message) from error
phoenix.session.client.TimeoutError: The request timed out. The timeout can be adjusted by passing a number of seconds to the `timeout` parameter and can be disabled altogether by passing `None`.

Suhrullekha C.
·
I've provided the api key in the header and the collector endpoint which is "https://app.phoenix.arize.com/s/<instancename>
Suhrullekha C.
·
Arize Phoenix is not running on https://app.phoenix.arize.com/s/... Launch Phoenix with `import phoenix as px; px.launch_app()` Traceback (most recent call last): File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 27, in <module> main() ~~~~^^ File "c:\Users\T767303\Desktop\Rapids Repos\phoenix\import_phoenix.py", line 20, in main phoenix_df = px.Client().get_spans_dataframe(project_identifier="fgjgjlljklk") TypeError: TraceDataExtractor.get_spans_dataframe() got an unexpected keyword argument 'project_identifier' 🔒[private user]
Suhrullekha C.
·
I'm not sure, how can i ensure that?
Suhrullekha C.
·
this takes me to a different page. I dont see my projects if i log in through that
Suhrullekha C.
·
I use the same website and when I log in using the welcome email link i recieved I'm able to see my projects. I just noticed this link also has phoenix cloud on it. Not sure why the link u sent me takes me to antother link with none of my projects even though I logged in using the same account
Suhrullekha C.
·
I actually got a sign up link from my team which created this and added all the projects in
Suhrullekha C.
·
the link has /s/ in it