Ben E.

Another question, if I may 🙂 I'm running Phoenix on a GCP VM with a separate Postgres instance backing it up. I'm running a benchmark of ~300 queries against my system, each such query runs prompts that total in the hundreds of thousands of tokens (I had a "session" with more than 7,000 spans). I registered Phoenix using register(project_name="mabool", batch=True, protocol="http/protobuf", auto_instrument=True) I'm using HTTP because gRPC seems to choke at some point (something with "resource exhausted" from otel, couldn't find something about it online). My question is: using the above config, is it normal to see traces pop up in the UI hours after a query ran? I'm asking because I thought some queries completely failed, so I killed my app, and hours later these sessions appeared in the UI. The "Stream" option is turned on, by the way. Thanks again for the help and for Phoenix, the self-hosting is amazing DX-wise 🤩 🙏 EDIT: I forgot to mention: my system runs on the same GCP VM that Phoenix runs on, I don't know how much it matters, i.e., how resource-intensive Phoenix is (my system is mainly IO-bound with all the network calls, not much CPU/RAM usage).

Hello 🙂 What would be the most efficient way to get token breakdown (completion, prompt, total, reasoning) by model for a specific session? Currently, I'm doing:

df = px.Client().get_spans_dataframe('session.id == "my-session-id"', project_name="my-project", limit=10000, timeout=30)

and then:

def get_token_breakdown(df):
    return (
        df.groupby("attributes.llm.model_name")
            .agg(
                {
                    "attributes.llm.token_count.total": "sum",
                    "attributes.llm.token_count.prompt": "sum",
                    "attributes.llm.token_count.completion": "sum",
                    "attributes.llm.token_count.completion_details.reasoning": "sum",
                }
            )
            .rename(
                columns={
                    "attributes.llm.token_count.total": "total",
                    "attributes.llm.token_count.prompt": "prompt",
                    "attributes.llm.token_count.completion": "completion",
                    "attributes.llm.token_count.completion_details.reasoning": "reasoning",
                }
            )
            .transpose()
            .to_dict()
    )

and 30 seconds timeout is definitely needed here. Is there a more performant way? Thanks! 🙏

Hi Arize community! I'm Ben, I work at Ai2 Israel on Paper Finder, and got here after hearing about Phoenix in Hamel & Shreya's evals course. Hoping to get to know Phoenix better than I what I already learned 🙂