Troubleshooting 500 Internal Server Error in Phoenix API Usage

Troubleshooting 500 Internal Server Error in Phoenix API Usage | Arize AI Community

14 comments

· Sorted by Oldest

RunLLM
APP
·
The 500 Internal Server Error you are encountering may be related to inconsistently typed metadata, which can cause issues when retrieving traces or spans using the Phoenix client, as noted in this GitHub issue. A quick fix is to filter out problematic metadata before querying spans. Additionally, ensure that your API key and endpoint are correctly set up in the environment variables.Would you like more detail on how to implement the filtering or troubleshoot further? Sources: - GitHub Issue 4420
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
Hey 🔒[private user], welcome back! 🔒[private user] can you take a look here?
Sean L.
·
Oh okay I'll try again
Sean L.
·
Thanks!!

Sean L.

🔒[private user] I was able to reproduce it from local. (ran phoenix as a docker container) and found this in the logs

pyarrow.lib.ArrowInvalid: ('cannot mix struct and non-struct, non-null values', 'Conversion failed for column attributes.llm.prompt_template.variables with type object')

Xander S.
·
Hey Sean L., how did this data get produced?
Xander S.
·
This is likely related to this issue, and there is a fix in progress. Sean L. If you can share a code snippet to reproduce, that would be helpful.

Sean L.

I don’t think I did anything particularly different from the tutorial. I tried both https://app.phoenix.arize.com and docker image (5.8.0). in both cases, the spans appear as expected in the browser, but when attempting to download it as df it gives me 500 error.

# Writing Span
import os
from langchain_anthropic import ChatAnthropic
from openinference.instrumentation.langchain import LangChainInstrumentor
from phoenix.otel import register

trace_provider = register(
    endpoint="http://localhost:4317"
)
LangChainInstrumentor().instrument(trace_provider=trace_provider)

llm = ChatAnthropic(
    model="claude-3-5-sonnet-latest",
    api_key=ANTHROPIC_API_KEY
)

llm.invoke("hello")

# Reading Span
# arize-phoenix==5.8.0
import phoenix as px

df = px.Client(
    endpoint="http://localhost:6006",
).get_spans_dataframe()

# Error Message
HTTPStatusError: Server error '500 Internal Server Error' for url 'http://localhost:6006/v1/spans?project_name=default&project-name=default'

Xander S.
·
thanks Sean L., we will test out the current fix to ensure it resolves the issue for you
Xander S.
·
Sean L. This issue should be resolved in arize-phoenix==5.9.0
Sean L.
·
It’s working! Thank you very much!
Sean L.
·
for the hosted phoenix, I had to clear the data. guess it’s the malformatted data that got in before the patch caused error. once I giving it new data, it’s responding as expected
Sean L.
·
Always impressed with the promptness of the team. Kudos 👏 👏👏
🙌1
Xander S.
·
forgot to mention, you also need to update your client version of arize-phoenix

RunLLM
APP
·
The 500 Internal Server Error you are encountering may be related to inconsistently typed metadata, which can cause issues when retrieving traces or spans using the Phoenix client, as noted in this GitHub issue. A quick fix is to filter out problematic metadata before querying spans. Additionally, ensure that your API key and endpoint are correctly set up in the environment variables.Would you like more detail on how to implement the filtering or troubleshoot further? Sources: - GitHub Issue 4420
💡 Hint: Mention RunLLM in the thread for followups.
Xander S.
·
Hey 🔒[private user], welcome back! 🔒[private user] can you take a look here?
Sean L.
·
Oh okay I'll try again
Sean L.
·
Thanks!!

Sean L.

🔒[private user] I was able to reproduce it from local. (ran phoenix as a docker container) and found this in the logs

pyarrow.lib.ArrowInvalid: ('cannot mix struct and non-struct, non-null values', 'Conversion failed for column attributes.llm.prompt_template.variables with type object')

Xander S.
·
Hey Sean L., how did this data get produced?
Xander S.
·
This is likely related to this issue, and there is a fix in progress. Sean L. If you can share a code snippet to reproduce, that would be helpful.

Sean L.

# Writing Span
import os
from langchain_anthropic import ChatAnthropic
from openinference.instrumentation.langchain import LangChainInstrumentor
from phoenix.otel import register

trace_provider = register(
    endpoint="http://localhost:4317"
)
LangChainInstrumentor().instrument(trace_provider=trace_provider)

llm = ChatAnthropic(
    model="claude-3-5-sonnet-latest",
    api_key=ANTHROPIC_API_KEY
)

llm.invoke("hello")

# Reading Span
# arize-phoenix==5.8.0
import phoenix as px

df = px.Client(
    endpoint="http://localhost:6006",
).get_spans_dataframe()

# Error Message
HTTPStatusError: Server error '500 Internal Server Error' for url 'http://localhost:6006/v1/spans?project_name=default&project-name=default'

Xander S.
·
thanks Sean L., we will test out the current fix to ensure it resolves the issue for you
Xander S.
·
Sean L. This issue should be resolved in arize-phoenix==5.9.0
Sean L.
·
It’s working! Thank you very much!
Sean L.
·
for the hosted phoenix, I had to clear the data. guess it’s the malformatted data that got in before the patch caused error. once I giving it new data, it’s responding as expected
Sean L.
·
Always impressed with the promptness of the team. Kudos 👏 👏👏
🙌1
Xander S.
·
forgot to mention, you also need to update your client version of arize-phoenix