Adding Custom Spans in LangChain Traces with Auto-instrumentation

Gaurav · 2025-05-02T10:12:47.967Z

How can I add custom spans inside Auto-instrumentation Langchain traces using LangChainInstrumentor.instrument(). It should be as nested span under current span which has been generated by autoinstrumenting the chain.

Adding Custom Spans in LangChain Traces with Auto-instrumentation | Arize AI Community

RunLLM
APP
·
You cannot directly nest a manual span under an auto-instrumented LangChain span using standard OpenTelemetry context propagation, because LangChain's callback-based tracing is isolated from the usual OTEL context. Manual spans created with `tracer.start_as_current_span` will not automatically appear as children of LangChain's auto-instrumented spans, since the context is not shared between them. To achieve true nesting, you would need to implement a custom callback or modify the callback manager to insert your custom span at the right point in the LangChain execution, similar to how LangChain attaches its own callbacks (<https://arize-ai.slack.com/archives/C04R3GXC8HK/p1716248614.485509|slack reference>). For a workaround, you can create a custom retriever or chain component and wire in your own callback handler, following the LangChain pattern. This allows your custom logic to participate in the same callback system and thus be properly nested in the trace tree (<https://arize-ai.slack.com/archives/C04R3GXC8HK/p1716248614.485509|slack reference>). Would you like a code example of how to do this? Sources: - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1716248614.485509|Slack: Manual span context with LangChain auto-instrumentation>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM Can you give example for custom retriever or chain component with integration in callback handler
RunLLM
APP
·
Certainly! To ensure your custom retriever is traced and its spans are nested under the auto-instrumented chain, inherit from `BaseRetriever` and let LangChain's callback system handle the tracing. Here’s a minimal example using `LangChainInstrumentor().instrument()` and a custom retriever (<https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-langchain/examples/custom_retriever.py|example>, <https://docs.arize.com/phoenix/tracing/concepts-tracing/faqs-tracing#%3AR4cd9uucqfkvfa%3A|docs>): ``` from typing import List from langchain_core.callbacks import CallbackManagerForRetrieverRun from langchain_core.retrievers import BaseRetriever, Document from openinference.instrumentation.langchain import LangChainInstrumentor from opentelemetry.sdk import trace as trace_sdk from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor tracer_provider = trace_sdk.TracerProvider() tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter())) LangChainInstrumentor().instrument(tracer_provider=tracer_provider) class CustomRetriever(BaseRetriever): k: int def _get_relevant_documents(self, query: str, *, run_manager: CallbackManagerForRetrieverRun) -> List[Document]: return [Document(page_content=f"dummy content at {i}") for i in range(self.k)] retriever = CustomRetriever(k=3) documents = retriever.invoke("what is the meaning of life?") print(documents) ``` This ensures your custom retriever's spans are auto-nested under the parent chain span. Would you like more detail on customizing the callback handler? Sources: - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-langchain/examples/custom_retriever.py|Custom retriever example> - <https://docs.arize.com/phoenix/tracing/concepts-tracing/faqs-tracing#%3AR4cd9uucqfkvfa%3A|Phoenix tracing FAQ: custom retriever>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM can it be done for custom rereanker too?
RunLLM
APP
·
Yes, you can instrument a custom reranker in the same way: inherit from the appropriate LangChain base class (e.g., implement as a retriever or post-processor), and ensure it is invoked within the LangChain callback system. This will allow Phoenix's auto-instrumentation to capture and nest reranker spans (<https://github.com/Arize-ai/phoenix/issues/1153|source>, <https://github.com/Arize-ai/phoenix/pull/1588|source>). However, note that some rerankers (like LlamaIndex's node post-processors) may not expose pre/post reranking data to the callback, so span detail may be limited. Would you like a code example for a custom reranker with tracing? Sources: - <https://github.com/Arize-ai/phoenix/issues/1153|Phoenix issue: reranker tracing> - <https://github.com/Arize-ai/phoenix/pull/1588|Phoenix PR: reranker span kind>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM also now I am using Manual instrumentation in my project but the spans are logged separate for different functions instead of nest spans. How can I achieve that.
RunLLM
APP
·
To nest manual spans, you must create child spans within the context of a parent span using a `with tracer.start_as_current_span("parent") as parent:` block, and then start child spans inside that block. This ensures the child spans are properly nested under the parent in the trace hierarchy (<https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|manual instrumentation guide>, <https://docs.arize.com/phoenix/tracing/how-to-tracing/setup-tracing/custom-spans#B%3A3|docs>). Example: ``` with tracer.start_as_current_span("parent") as parent: # Parent span logic with tracer.start_as_current_span("child") as child: # Child span logic ``` If your functions are in different code paths, pass the parent span context explicitly using `set_span_in_context` (<https://arize-ai.slack.com/archives/C04R3GXC8HK/p1715994066.749849|slack example>). Would you like more detail on context propagation? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|Manual instrumentation guide> - <https://docs.arize.com/phoenix/tracing/how-to-tracing/setup-tracing/custom-spans#B%3A3|Phoenix docs: custom spans> - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1715994066.749849|Slack: parent/child span context>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM can you give code example wrt to set_span_in_context
RunLLM
APP
·
Certainly! Here’s a minimal example showing how to nest a child span under a parent span using `set_span_in_context` to propagate the parent context (<https://arize-ai.slack.com/archives/C04R3GXC8HK/p1715994066.749849|slack example>): ``` from opentelemetry import trace from opentelemetry.trace import set_span_in_context, SpanKind tracer = trace.get_tracer(__name__) def create_child_span(tracer, parent_span): parent_context = set_span_in_context(parent_span) child_span = tracer.start_span( "child_span", kind=SpanKind.INTERNAL, context=parent_context, ) # Do work in child span child_span.end() with tracer.start_as_current_span("parent_span") as parent_span: create_child_span(tracer, parent_span) ``` This ensures the child span is nested under the parent span in the trace. Would you like more detail or a multi-function example? Sources: - <https://arize-ai.slack.com/archives/C04R3GXC8HK/p1715994066.749849|Slack: set_span_in_context example>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM I want to use manual instrumentation for using @tracer.llm decorator. How can I get input output and invocaton parameters like we get in Auto instrumentation.
RunLLM
APP
·
When using manual instrumentation with the `@tracer.llm` decorator, you must explicitly pass the input, output, and invocation parameters to the span, as these are not automatically extracted like in auto-instrumentation. You can do this by setting attributes inside your decorated function using the span object (<https://docs.arize.com/phoenix/tracing/how-to-tracing/instrument-python#llms|docs>, <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|manual instrumentation guide>). Example: ``` @tracer.llm def my_llm_call(messages, model, temperature): # ... call your LLM here ... response = call_llm(messages, model, temperature) span = trace.get_current_span() span.set_attribute("llm.input_messages", messages) span.set_attribute("llm.invocation_parameters", {"model": model, "temperature": temperature}) span.set_attribute("output.value", response) return response ``` Would you like more detail or a more advanced example? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|Manual instrumentation guide>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM can you give me sample code for passing to reranker along with attribute for input and output documents in reranker span
RunLLM
APP
·
Certainly! For manual instrumentation of a reranker span, you should use the OpenInference semantic conventions and set the input and output documents as span attributes. Here’s a minimal example (<https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|manual instrumentation guide>, <https://github.com/Arize-ai/phoenix/issues/1153|reranker attribute reference>): ``` from opentelemetry import trace from openinference.semconv.trace import RerankerAttributes, OpenInferenceSpanKindValues tracer = trace.get_tracer(__name__) input_docs = [{"content": "doc1"}, {"content": "doc2"}] output_docs = [{"content": "doc2"}, {"content": "doc1"}] with tracer.start_as_current_span( "reranker", attributes={ "openinference.span.kind": OpenInferenceSpanKindValues.RERANKER.value, RerankerAttributes.RERANKER_INPUT_DOCUMENTS: input_docs, RerankerAttributes.RERANKER_OUTPUT_DOCUMENTS: output_docs, } ) as span: # Reranking logic here pass ``` This will log a reranker span with input and output documents as attributes. Would you like more detail or a multi-step example? Sources: - <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/custom-spans.md|Manual instrumentation guide> - <https://github.com/Arize-ai/phoenix/issues/1153|Reranker attribute reference>
💡 Hint: Mention RunLLM in the thread for followups.
Gaurav
·
RunLLM Its asking for flattening of list I have below docuemnts list [{"document.id": "1", "document.score": 0.9, "document.content": "..."}] How Can I pass it to output documents
RunLLM
APP
·
You should flatten the list of document objects using the OpenInference conventions before setting them as span attributes. Use a utility like `_flatten` to convert nested lists/dicts into the required flat key-value pairs (<https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md|spec example>, <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-groq/src/openinference/instrumentation/groq/_wrappers.py|flatten utility reference>). Example: ``` from openinference.semconv.trace import RerankerAttributes from openinference.instrumentation.groq._wrappers import _flatten output_docs = [{"document.id": "1", "document.score": 0.9, "document.content": "..."}] span.set_attributes(dict(_flatten({RerankerAttributes.RERANKER_OUTPUT_DOCUMENTS: output_docs}))) ``` This will flatten your list so Phoenix can ingest it. Would you like more detail or a full reranker span example? Sources: - <https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md|Semantic conventions: flattening> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-groq/src/openinference/instrumentation/groq/_wrappers.py|Flatten utility example>
💡 Hint: Mention RunLLM in the thread for followups.

71 comments