Noah S.

Commented on LlamaIndex Instrumentation Issues in Docker with O...·Posted inPhoenix Support

Sweet, this is a great bandaid thank you! I guess I will have to port over our pipelines to workflows. Thanks 🔒[private user] and 🔒[private user], that was fast!

Commented on LlamaIndex Instrumentation Issues in Docker with O...·Posted inPhoenix Support

Noah S.

Yeah that works!

Posted in Phoenix Support·

Noah S.

LlamaIndex Instrumentation Issues in Docker with OpenTelemetry

RunLLM I do have a quick question. I'm trying to hook up my older llamaindex project, I've updated llamaindex to 0.10.43 and using the LlamaIndexInstrumentor, but it's not sending anything to the running docker container. Do LlamaIndex Pipelines not support the new instrumentation? Do I need to move to workflows?

openinference-instrumentation-llama-index = "^3.0.3"
arize-phoenix-otel = "^0.6.1"
arize-phoenix = "^4.32.0"
opentelemetry-sdk = "^1.27.0"
opentelemetry-exporter-otlp = "^1.27.0"
opentelemetry-api = "^1.27.0"
opentelemetry-instrumentation-fastapi = "^0.48b0"

Code

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from phoenix.otel import register

tracer_provider = register(
    endpoint="http://localhost:6006/v1/traces",
)
LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

The code is written in a FastAPI file so should be triggered on app startup. Trying to run via uvicorn, which also maybe has some port wiring extra steps?

poetry run uvicorn api.app.main:app --reload

The API layer directs from endpoint -> Semantic Router, which invokes a LLamaIndex Pipeline. I know they moved to Workflows recently but was trying to avoid having to rewrite it all for as long as possible

5Comments

Commented on Challenges with Bedrock LLM Tracing in LlamaIndex...·Posted inPhoenix Support

Noah S.

Actually I found 🔒[private user]’s question immediately above mine to be super helpful. That seems to be how I would invoke the custom Spans, which I guess I can just override every component in my pipeline to leverage. I'm looking to see if there is a way to combine the LlamaIndex Instrumentor and custom spans so that I don't need to do it for spans that are already handled

Commented on Challenges with Bedrock LLM Tracing in LlamaIndex...·Posted inPhoenix Support

Noah S.

Yes, just one span for the LLM call would be ideal. It does make filtering difficult as they both have the same id. I am also not really tied to the legacy callback system, all I really want is the ability to run a QueryPipeline and see the traces so that I can then create an eval dataset and test different prompts/models against hallucinations, retrieval, etc.

Posted in Phoenix Support·

Noah S.

Challenges with Bedrock LLM Tracing in LlamaIndex QueryPipeline

I'm back. For whatever reason, when I invoke a Bedrock LLM object as part of a LlamaIndex QueryPipeline, it for some reason invokes a trace on both prompt_to_messages and the original prompt itself. To get around this, I tried to just define a CustomQueryComponent and invoke the LLM .complete() explicitly in the run_component method. However, this caused the trace to be dropped entirely. I reached out to the LlamaIndex team and they were unsure if Phoenix supports the ability to tag custom components. But this is confusing to me, I don't want to have to deal with the native LlamaIndex object generating two traces, but if I can't add a trace label to a custom component I don't think I have a choice. Anyone have any ideas?

4Comments

Commented on Troubleshooting ValueError in Evaluating DataFrame...·Posted inPhoenix Support

Noah S.

Thank you!

Commented on Troubleshooting ValueError in Evaluating DataFrame...·Posted inPhoenix Support

Noah S.

Yeah this is returned by the get_qa_with_reference

Commented on Troubleshooting ValueError in Evaluating DataFrame...·Posted inPhoenix Support

Noah S.

Sorry I can't share more without showing data, but because of each span being an individual step in the pipeline, it's picking up output from each component in the QueryPipeline as output

Commented on Troubleshooting ValueError in Evaluating DataFrame...·Posted inPhoenix Support

Noah S.

That worked! But because of the tracing being out of whack, it's returning intermediary steps as output which I don't want. Do you know how I can maybe not use the LlamaIndex to set phoenix as a handler so I can get back to a single chain for tracing?