Sagar D.

Commented on Improvements in Arize: Easier Evals and Updated Ch...·Posted inArize AX Releases

Hi! I would love to test the evals in the new playground. However even with the latest 8.25.0 version, I don't see the Evals and Tasks on my left nav. This is the snippet from my docker-compose. Screenshots from my locally stood up Phoenix also attached. can you please tell me which version has the new Evals please? As always appreciate your help!

  phoenix:
    container_name: phoenix
    image: arizephoenix/phoenix:version-8.25.0
    attach: false # remove if you want to see logs from phoenix
    profiles:
      - monitoring
    ports:
      - 6006:6006
....

Commented on Bugs and Improvements for Authenticated Phoenix In...·Posted inPhoenix Support

Sagar D.

Commented on the PR as suggested ✅

Commented on Bugs and Improvements for Authenticated Phoenix In...·Posted inPhoenix Support

Sagar D.

I am at work. As soon as I get back home tonight.

Commented on Bugs and Improvements for Authenticated Phoenix In...·Posted inPhoenix Support

Sagar D.

🔒[private user] https://github.com/Arize-ai/phoenix/pull/6834 created for #2 . It was created from my personal account as my work account does not support contribution to outside projects. Please review and lmk.

Commented on Bugs and Improvements for Authenticated Phoenix In...·Posted inPhoenix Support

Sagar D.

I can help with #2 so lmk

Posted in Phoenix Support·

Sagar D.

Bugs and Improvements for Authenticated Phoenix Instance Reporting

I have a couple of bugs/improvements to report:

1.
Log Evaluation results to Authenticated Phoenix instance - The documentation at https://docs.arize.com/phoenix/tracing/how-to-tracing/feedback-and-annotations/llm-evaluations#connect-to-phoenix suggests adding os.environ["PHOENIX_CLIENT_HEADERS"] = f"api_key=...", but this does not work because of the following code at http://github.com/Arize-ai/phoenix/blob/main/src/phoenix/config.py#L717 . The function parse_env_headers parses header to be a dict like so {"api_key": ...} , which does not work as authorization header. Following this code, I used os.environ["PHOENIX_API_KEY"]=...which worked. Suggest changing the documentation to reflect adding PHOENIX_API_KEY instead of PHOENIX_CLIENT_HEADERS

def get_env_client_headers() -> dict[str, str]:
    headers = parse_env_headers(getenv(ENV_PHOENIX_CLIENT_HEADERS))
    if (api_key := get_env_phoenix_api_key()) and "authorization" not in [
        k.lower() for k in headers
    ]:
        headers["Authorization"] = f"Bearer {api_key}"
    return headers

2. Add a timeout param to the utility function get_qa_with_reference https://docs.arize.com/phoenix/tracing/how-to-tracing/importing-and-exporting-traces/extract-data-from-spans#q-and-a-on-retrieved-data - The 5 seconds default timeout is not enough to query on large tables, so recommend adding the timeout param which can then be passed to query_spans

5Comments

Commented on Issue with Tag Visibility for Non-LLM Span Kinds i...·Posted inPhoenix Support

Sagar D.

Thanks a lot Xander S.! That worked!!

Commented on Issue with Tag Visibility for Non-LLM Span Kinds i...·Posted inPhoenix Support

Sagar D.

That worked, thanks! But an interesting observation: The tags and metadata are not showing for the parent span process_document_ingestion, but it is showing for the child span autotag_document Here is the pseudo code. Am I missing something?

meta_px = {"tag_1": "ingestion", "tag_2": ingestion_job.id}
with tracer.start_as_current_span("process_document_ingestion") as span, using_tags(list(meta_px.values())), using_metadata(meta_px):
    span.set_attribute(SpanAttributes.OPENINFERENCE_SPAN_KIND, "CHAIN")
    span.set_attribute(SpanAttributes.INPUT_VALUE, ingestion_job.prodigy_document.name)
    await autotag_document(ingestion_job.prodigy_document, ingestion_job.case_context)


async def autotag_document(prodigy_document: ProdigyDocument):
   
    with tracer.start_as_current_span("autotag_document") as span:
        span.set_attribute(SpanAttributes.OPENINFERENCE_SPAN_KIND, "AGENT")
        span.set_attribute(SpanAttributes.INPUT_VALUE, prodigy_document.name)

Commented on Issue with Tag Visibility for Non-LLM Span Kinds i...·Posted inPhoenix Support

Sagar D.

Disregard the above example please. Here is a cleaner code. The span is being set with the right span-kind and input value. But neither the tag nor the metadata are appearing.

from openinference import semconv
from openinference.semconv import resource
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry import trace
from opentelemetry.trace import Tracer
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor


resource = Resource(
    attributes={semconv.resource.ResourceAttributes.PROJECT_NAME: 'phoenix'}
)
tracer_provider = TracerProvider(resource=resource)
trace.set_tracer_provider(tracer_provider)
span_exporter = OTLPSpanExporter(
    endpoint='http://localhost:6006/v1/traces',
)
simple_span_processor = SimpleSpanProcessor(span_exporter=span_exporter)
trace.get_tracer_provider().add_span_processor(simple_span_processor)

tracer = trace.get_tracer(__name__)

meta_px = {"tag_1": "ingestion", "tag_2": "tag2"}

with tracer.start_as_current_span("process_document_ingestion") as span, using_tags(list(meta_px.values())), using_metadata(meta_px):
    span.set_attribute(SpanAttributes.OPENINFERENCE_SPAN_KIND, "CHAIN")
    span.set_attribute(SpanAttributes.INPUT_VALUE, "document_name")

Commented on Issue with Tag Visibility for Non-LLM Span Kinds i...·Posted inPhoenix Support

Sagar D.

Unfortunately, it seems to be the same issue for using metadata too. It is appearing only for span-kind = llm and not for other span kinds. This is the code I am using.

with using_tags(['chat-api']), using_metadata({"tag-1": "tag1"}):
   ....