RunLLM

The "Email in Use" error occurs because Phoenix enforces a uniqueness constraint on user email addresses. If a user is deleted, but their database record is only soft-deleted (i.e., marked with a deleted_at timestamp), the email remains reserved and cannot be reused for new accounts. This is especially problematic with OAuth2/Keycloak, as users cannot re-register with the same email until the record is fully removed from the database. The issue is resolved temporarily when you manually delete the user record, but it will recur if the deletion process only marks the user as deleted instead of actually removing the record from the database. Recent Phoenix versions have addressed this by fully deleting user tuples instead of soft-deleting them, which resolves the email reuse problem for OAuth2 and SSO integrations. See the discussion and resolution in <https://github.com/Arize-ai/phoenix/issues/4676|this issue>. If you are still experiencing this issue on 12.35.0, it may indicate that your deployment is not applying the latest database migration or that some users are still being soft-deleted. Ensure your database schema is up to date and that user deletions are fully removing records, not just marking them as deleted. If the problem persists, check for any customizations or older migration scripts that might be interfering with the deletion logic. Would you like more detail on how to verify or update your database schema and user deletion behavior? Sources: - <https://github.com/Arize-ai/phoenix/issues/4676|https://github.com/Arize-ai/phoenix/issues/4676>

Your configuration format using `PHOENIX_POSTGRES_HOST`, `PHOENIX_POSTGRES_USER`, `PHOENIX_POSTGRES_PASSWORD`, and `PHOENIX_POSTGRES_DB` *is still supported in v14*. The migration docs confirm that `PHOENIX_SQL_DATABASE_URL` continues to work, but the environment variable approach you are using is also explicitly documented and supported for PostgreSQL connections in v14 and later. You can use either the connection string (`PHOENIX_SQL_DATABASE_URL`) or the individual environment variables for database configuration—no changes are required for your setup when upgrading from v13 to v14. See <https://arize.com/docs/phoenix/self-hosting/configuration#server-configuration|official configuration docs> and <https://github.com/Arize-ai/phoenix/blob/main/MIGRATION.md|migration notes> for confirmation. Would you like more detail on how Phoenix prioritizes these variables or example configurations? Sources: - <https://arize.com/docs/phoenix/self-hosting/configuration#server-configuration|Phoenix configuration docs> - <https://github.com/Arize-ai/phoenix/blob/main/MIGRATION.md|Phoenix migration guide>

Online Evals in Arize AX refer to real-time, automated evaluations of LLM outputs as traces are ingested—allowing for immediate scoring, alerting, and monitoring in production. Phoenix (the open-source, self-hosted version) currently does not support true online evals; evaluations must be run in batches after traces are collected, typically via scheduled jobs or manual triggers, rather than automatically as data arrives. This means Phoenix is limited to "offline" or "batch" evaluations, while Arize AX provides continuous, production-grade monitoring with threshold-based alerts and dashboards for evaluation metrics like faithfulness or correctness as soon as new data is available. If you need real-time, always-on evaluation and alerting for production LLM traffic, this is only available in Arize AX. For most experimentation, prompt engineering, and batch evaluation use cases, Phoenix is sufficient and much easier to self-host. For more detail, see <https://arize.com/docs/phoenix/resources/frequently-asked-questions/langfuse-alternative-arize-phoenix-vs-langfuse-key-differences#ease-of-self-hosting|Phoenix vs. Langfuse/Arize AX comparison> and <https://arize.com/docs/phoenix/user-guide#panel-iterate-via-experiments-0|Phoenix User Guide>. Would you like a step-by-step breakdown of how evaluations work in Phoenix and how to set up batch evals? Sources: - <https://arize.com/docs/phoenix/resources/frequently-asked-questions/langfuse-alternative-arize-phoenix-vs-langfuse-key-differences#ease-of-self-hosting|Phoenix vs. Langfuse/Arize AX comparison> - <https://arize.com/docs/phoenix/user-guide#panel-iterate-via-experiments-0|Phoenix User Guide>

To populate columns like input, output, and tool calls in Phoenix, you need to use OpenInference semantic conventions and/or OpenInference instrumentors, not just generic OpenTelemetry tracing. Standard OTEL spans alone do not automatically capture LLM-specific attributes (inputs, outputs, tool calls, etc.) required by Phoenix’s UI. You should either use OpenInference instrumentor packages for your frameworks (e.g., OpenAI, LangChain, etc.) or manually set semantic attributes on your spans according to the OpenInference conventions <https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans|[Phoenix manual instrumentation guide>, <https://arize.com/docs/phoenix/sdk-api-reference#environment-variables|OpenInference docs>]. For Python, install the relevant OpenInference instrumentor (e.g., `pip install openinference-instrumentation-openai`) and use `phoenix.otel.register(auto_instrument=True)` to auto-instrument supported libraries, or use decorators/context managers like `@tracer.chain` and `span.set_input()/set_output()` for manual spans. For each span, ensure you set attributes such as `input.value`, `output.value`, and tool-specific attributes as described in the <https://docs.arize.com/phoenix/tracing/how-to-tracing/manual-instrumentation/custom-spans|manual instrumentation docs>. Without these, Phoenix cannot display the detailed columns in its UI. Would you like a code example for your specific framework or more detail on manual attribute setting? Sources: - <https://arize.com/docs/phoenix/sdk-api-reference#environment-variables|Phoenix SDK/API reference>

Yes, retries and context issues in agent workflows can cause wasted cost and debugging challenges. For example, in Phoenix, excessive context can exceed token limits, breaking coherence and causing agents to repeat requests or loop unnecessarily, which increases LLM/API costs and makes failures hard to trace. Poorly defined agent roles or missing termination conditions can also lead to agents continuing beyond logical endpoints, further compounding cost and complexity. These issues are common in multi-agent and long-running workflows, especially when context drift or bandwidth overload occurs, leading to coordination failures and unpredictable behavior. Real-world users have reported such problems, including runaway retries and loops due to misconfigured context or lack of explicit stop conditions, resulting in high costs and opaque trace logs that are difficult to debug and optimize according to <https://arize.com/docs/phoenix/prompt-engineering/concepts-prompts/context-engineering-basics|Phoenix context engineering documentation> and <https://arize.com/docs/phoenix/cookbook/agent-workflow-patterns/autogen#design-considerations-and-limitations|AutoGen agent workflow patterns>. Would you like a more detailed breakdown or a code-level example? Sources: - <https://arize.com/docs/phoenix/prompt-engineering/concepts-prompts/context-engineering-basics|Phoenix context engineering documentation> - <https://arize.com/docs/phoenix/cookbook/agent-workflow-patterns/autogen#design-considerations-and-limitations|AutoGen agent workflow patterns>

Phoenix is actively working toward Python 3.14 support. There are no known blockers for Phoenix itself, and a client update for Python 3.14 has already been merged (<https://github.com/Arize-ai/phoenix/pull/11830|PR #11830>). However, some dependencies—such as `sqlean-py`—have caused installation issues, and related OpenInference instrumentation packages are still updating their Python version requirements (<https://github.com/Arize-ai/phoenix/issues/10001|issue #10001>, <https://github.com/Arize-ai/openinference/issues/2285|issue #2285>). The Google ADK instrumentation package (`openinference-instrumentation-google-adk`) added Python 3.14 support in version 0.1.6 (<https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-google-adk/CHANGELOG.md|CHANGELOG.md>), so you should use at least this version. In summary, Python 3.14 support is landing across Phoenix and its dependencies, but you may still encounter issues with some packages until all dependencies are fully updated. There is no official timeline, but the process is ongoing and tracked in the linked issues and changelogs. Would you like a step-by-step guide for setting up Phoenix and Google ADK with Python 3.14? Sources: - <https://github.com/Arize-ai/phoenix/pull/11830|Phoenix PR: update client to python 3.14> - <https://github.com/Arize-ai/phoenix/issues/10001|Phoenix issue: Python 3.14 support> - <https://github.com/Arize-ai/openinference/issues/2285|OpenInference issue: Relax Python version requirements to support Python 3.14+> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-google-adk/CHANGELOG.md|openinference-instrumentation-google-adk CHANGELOG.md>

AgentExecutor traces are included in project metrics because Phoenix aggregates all ingested traces for a project by default, regardless of their origin or type. This design ensures that every trace—whether from LangChain’s AgentExecutor or Langflow’s native UUID trace—contributes to overall project statistics, providing a comprehensive view of all activity unless specifically filtered (<https://arize.com/docs/phoenix/integrations/platforms/langflow/langflow-tracing|source>, <https://arize.com/docs/phoenix/tracing/concepts-tracing#traces|source>). However, this can be misleading if different trace types represent different logical units or latencies, as their metrics may not be directly comparable. The rationale is to maximize observability and flexibility, but it requires users to apply filters or post-process data to focus on the most relevant trace types for their use case. There is ongoing discussion in the community about making project metrics filter-aware to address this confusion (<https://github.com/Arize-ai/phoenix/issues/5973|source>). Would you like more detail on best practices for handling mixed trace types in Phoenix? Sources: - <https://arize.com/docs/phoenix/integrations/platforms/langflow/langflow-tracing|LangFlow Tracing - Phoenix> - <https://arize.com/docs/phoenix/tracing/concepts-tracing#traces|https://arize.com/docs/phoenix/tracing/concepts-tracing#traces> - <https://github.com/Arize-ai/phoenix/issues/5973|https://github.com/Arize-ai/phoenix/issues/5973>

Yes, this is expected behavior: applying a filter in the Phoenix UI currently does not affect the project-level latency metrics (such as p50/p99) shown in the header; these metrics are calculated over all traces in the project, regardless of filters. Filters only affect the data shown in tables and charts, not the summary metrics at the top (<https://github.com/Arize-ai/phoenix/issues/5973|source>, <https://github.com/Arize-ai/phoenix/issues/9008|source>, <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1745696860.492159|source>). If you need latency metrics for a filtered subset, you must export the filtered data and compute the metrics externally, or use the GraphQL API, which may reflect filters more accurately than the UI header. Would you like more detail or a code example for exporting and calculating filtered latency? Sources: - <https://github.com/Arize-ai/phoenix/issues/5973|https://github.com/Arize-ai/phoenix/issues/5973> - <https://github.com/Arize-ai/phoenix/issues/9008|https://github.com/Arize-ai/phoenix/issues/9008> - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1745696860.492159|https://arize-ai.slack.com/archives/C04R3GXC8HK/p1745696860.492159>

Yes, it is expected that both AgentExecutor and UUID traces are included in Phoenix’s latency metrics by default, since Phoenix aggregates all traces ingested for a project unless you apply a filter. This can lead to mixed latency statistics if both trace types are present in your data. According to <https://arize.com/docs/phoenix/integrations/platforms/langflow/langflow-tracing|Phoenix documentation>, the AgentExecutor trace comes from Phoenix’s LangChain instrumentation, while the UUID trace is native to Langflow. To track latency using only UUID traces, you should apply a filter in the Phoenix UI or API that selects only the UUID traces (e.g., by trace name, span kind, or a unique attribute present in UUID traces). This will ensure that latency metrics (p50, p99, etc.) are calculated only for the traces that represent Langflow service requests, excluding AgentExecutor traces from the aggregation. Filtering by trace type or attribute is the recommended approach for targeted metrics in Phoenix (<https://arize.com/docs/phoenix/tracing/concepts-tracing/faqs-tracing|source>). Would you like more detail on how to set up such a filter in the Phoenix UI or via API? Sources: - <https://arize.com/docs/phoenix/integrations/platforms/langflow/langflow-tracing|LangFlow Tracing - Phoenix> - <https://arize.com/docs/phoenix/tracing/concepts-tracing/faqs-tracing|https://arize.com/docs/phoenix/tracing/concepts-tracing/faqs-tracing>

Phoenix is not optimized for very large-scale deployments (hundreds of millions of spans, multi-terabyte databases) in its open-source form. Common issues at this scale include slow queries, UI timeouts, and database contention, especially with large spans tables and during migrations. Recommended approaches include: (1) aggressive data retention policies to keep the database size manageable, (2) regular pruning of low-value spans, (3) scaling up database resources (CPU, memory, parallel workers), and (4) horizontal scaling of Phoenix containers behind a load balancer, all backed by a single Postgres instance. For even larger workloads or OLAP use cases, Arize AX (the enterprise product) uses a proprietary database designed for high-volume analytics, which is not available in open-source Phoenix. There is no documented support for sharding or distributed Postgres in open-source Phoenix, and performance bottlenecks are expected at this scale without moving to a more specialized backend or product. Recent improvements in Phoenix include UI lazy loading, pagination, and query optimizations, but these only partially mitigate issues with very large datasets. For production, it's recommended to monitor memory and disk usage, tune Postgres (indexes, query plans, connection pooling), and consider splitting workloads across multiple Phoenix instances if organizationally feasible. If you continue to experience performance issues, you may be hitting the architectural limits of open-source Phoenix and Postgres for this use case. Would you like more detail on specific tuning steps or architectural alternatives? Sources: - <https://arize.com/docs/phoenix/self-hosting/architecture|Phoenix architecture and scaling> - <https://arize.com/docs/phoenix/production-guide|Production guide: scaling and resource planning> - <https://github.com/Arize-ai/phoenix/issues/6201|UI slowdowns> - <https://github.com/Arize-ai/phoenix/issues/8420|database tuning>