Disabling Auto-Instrumentation for Llama Index in Your App

Disabling Auto-Instrumentation for Llama Index in Your App | Arize AI Community

12 comments

· Sorted by Oldest

Mikyo
·
Hi Gregory S. - thanks for reaching out. basically only want LLM spans would that be my understanding? In general we currently don't support turning off parts of your trace but we may have a few options depending on what LLMs you use - this wouldn't require llama-index instrumentation at all.
Mikyo
·
There might be a way actually - where you could not set a global handler but rather only pass our handler to synthesis.
Gregory S.
·
only want LLM spans
Kind of, but also not all LLM spans. Because with global instrumentation our custom evaluators based on LlamaIndex base evaluator class are also being collected as LLM spans.
Gregory S.
·
Could you explain what do you mean by our handler ?
Mikyo
·
I see I see, so there are some options for stopping instrumentation. Here's an example: https://docs.arize.com/phoenix/tracing/how-to-tracing/tracing-core-concepts#pause-tracing-using-context-manager
Mikyo
·
so if you wrap your evaluators with that context manager you won't get traces for those. Alternatively you can send it to a different project if you are using it in a notebook.
Gregory S.
·
I see. I’ll give it a try, thank you.
Mikyo
·
Could you explain what do you mean by our handler ?
Yes, so set_global_handler gets added to llama-index as something to receive callback info for every part of your llama-index engine. See (https://github.com/run-llama/llama_index/blob/fcdc55006055e114d4e6b606804d67bb9c27[…]/llama-index-core/llama_index/core/callbacks/global_handlers.py) However you can technically attach handlers on just the parts you care about.
Mikyo
·
You can find our handler under a package called openinference-instrumentation-llama-index And you can add that callback to just the parts you want:
from openinference.instrumentation.llama_index._callback import ( OpenInferenceTraceCallbackHandler, )
Mikyo
·
It's a bit more work but would look similar to this: https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinf[…]index/src/openinference/instrumentation/llama_index/__init__.py
Gregory S.
·
I see, thank you.
Mikyo
·
No problem. Let us know if you need any further support. Happy to take feature requests as well.
1

Mikyo
·
Hi Gregory S. - thanks for reaching out. basically only want LLM spans would that be my understanding? In general we currently don't support turning off parts of your trace but we may have a few options depending on what LLMs you use - this wouldn't require llama-index instrumentation at all.
Mikyo
·
There might be a way actually - where you could not set a global handler but rather only pass our handler to synthesis.
Gregory S.
·
only want LLM spans
Kind of, but also not all LLM spans. Because with global instrumentation our custom evaluators based on LlamaIndex base evaluator class are also being collected as LLM spans.
Gregory S.
·
Could you explain what do you mean by our handler ?
Mikyo
·
I see I see, so there are some options for stopping instrumentation. Here's an example: https://docs.arize.com/phoenix/tracing/how-to-tracing/tracing-core-concepts#pause-tracing-using-context-manager
Mikyo
·
so if you wrap your evaluators with that context manager you won't get traces for those. Alternatively you can send it to a different project if you are using it in a notebook.
Gregory S.
·
I see. I’ll give it a try, thank you.
Mikyo
·
Could you explain what do you mean by our handler ?
Yes, so set_global_handler gets added to llama-index as something to receive callback info for every part of your llama-index engine. See (https://github.com/run-llama/llama_index/blob/fcdc55006055e114d4e6b606804d67bb9c27[…]/llama-index-core/llama_index/core/callbacks/global_handlers.py) However you can technically attach handlers on just the parts you care about.
Mikyo
·
You can find our handler under a package called openinference-instrumentation-llama-index And you can add that callback to just the parts you want:
from openinference.instrumentation.llama_index._callback import ( OpenInferenceTraceCallbackHandler, )
Mikyo
·
It's a bit more work but would look similar to this: https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinf[…]index/src/openinference/instrumentation/llama_index/__init__.py
Gregory S.
·
I see, thank you.
Mikyo
·
No problem. Let us know if you need any further support. Happy to take feature requests as well.
1