In-Memory Span Collector for Unit Tests in OpenInference

Ben M.

·Sep 03, 2025 01:16 PM

Does openinference have an in memory span collector that I can use while running unit tests?

10 comments

· Sorted by Oldest

RunLLM
APP
·
Yes, OpenInference uses the OpenTelemetry `InMemorySpanExporter` for in-memory span collection during unit tests. This is shown in multiple test setups, where `InMemorySpanExporter` is used to collect and inspect spans without exporting them externally. You can use this exporter with your tracer provider in your test fixtures or setup code to capture spans in memory for assertions and validation purposes. Example usage can be found in the OpenInference test files, such as: ``` from opentelemetry.sdk.trace.export import InMemorySpanExporter, SimpleSpanProcessor from opentelemetry.sdk.trace import TracerProvider in_memory_span_exporter = InMemorySpanExporter() tracer_provider = TracerProvider() tracer_provider.add_span_processor(SimpleSpanProcessor(in_memory_span_exporter)) ``` According to <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|conftest.py (litellm)>, <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai/tests/openinference/instrumentation/openai/conftest.py|conftest.py (openai)>, and <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai-agents/tests/conftest.py|conftest.py (openai-agents)>. Would you like more detail or a full test example? Sources: - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|conftest.py (litellm)> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai/tests/openinference/instrumentation/openai/conftest.py|conftest.py (openai)> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai-agents/tests/conftest.py|conftest.py (openai-agents)>
💡 Hint: Mention RunLLM in the thread for followups.
Ben M.
·
RunLLM can you show me what packages I need? opentelemetry.sdk seems to not have those files when just installing arize-phoenix-otel
RunLLM
APP
·
You need to install the `opentelemetry-sdk` package directly, as `arize-phoenix-otel` does not include it as a dependency. The `InMemorySpanExporter` and `SimpleSpanProcessor` are provided by `opentelemetry-sdk`, not by `arize-phoenix-otel` itself. Install with: ``` pip install opentelemetry-sdk ``` This will provide the necessary modules for in-memory span collection in your tests. See <https://arize.com/docs/phoenix/sdk-api-reference/python-packages/arize-phoenix-otel|arize-phoenix-otel docs> and <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|OpenInference test setup>. Would you like a full requirements list for a typical test environment? Sources: - <https://arize.com/docs/phoenix/sdk-api-reference/python-packages/arize-phoenix-otel|arize-phoenix-otel docs>
💡 Hint: Mention RunLLM in the thread for followups.
Ben M.
·
Unfortunately that doesn’t seem to work. I made sure that opentelemetry-sdk was the latest version according to pypi
Xander S.
·
Hey Ben M., it might be worth checking out our test suite for one our auto-instrumentors. https://github.com/Arize-ai/openinference/blob/72a6f06b4ea992a13ba0258ca593a1f7769[…]n-openai/tests/openinference/instrumentation/openai/conftest.py
Xander S.
·
https://github.com/Arize-ai/openinference/blob/72a6f06b4ea992a13ba0258ca593a1f7769[…]on-anthropic/tests/openinference/anthropic/test_instrumentor.py

Ben M.

Thank you, Xander S.! That fixed me up. This is what I ended up with

if config.env != "test":
    tracer = register(
        project_name=config.phoenix_project_name,
        endpoint=f"{config.phoenix_url}/v1/traces",
        headers={"Authorization": f"Bearer {config.phoenix_api_key}"},
        auto_instrument=True,
        verbose=False,
        batch=True,
    ).get_tracer(__name__)
else:
    from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
        InMemorySpanExporter,
    )
    from opentelemetry.sdk.trace.export import SimpleSpanProcessor
    from openinference.instrumentation import TracerProvider

    in_memory_span_exporter = InMemorySpanExporter()
    tracer_provider = TracerProvider()
    tracer_provider.add_span_processor(SimpleSpanProcessor(in_memory_span_exporter))
    tracer = tracer_provider.get_tracer(__name__)

👍1

💡1

RunLLM
APP
·
We've found some places in your existing documentation that contradict the support staff knowledge you've provided! See the docs <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/setup-tracing/custom-spans.md|here> and <https://github.com/Arize-ai/phoenix/blob/main/docs/section-sdk-api-reference/python-packages/arize-phoenix-otel.md|here> Please go <https://app.runllm.com/knowledge/issues|here> to view and resolve all the inconsistencies found for this assistant.
Ben M.
·
Xander S. what if register allowed an in_memory kwarg that would automatically use an InMemorySpanExporter when it is passed True ? I could submit a PR this week if this is something the Arize team is open to
Xander S.
·
Hey Ben M., I would probably avoid that since register is intended for getting up and running in notebooks rather than writing unit tests. In a unit test environment, I would recommend setting up OTel primitives as in the snippet above.
👍1

RunLLM
APP
·
Yes, OpenInference uses the OpenTelemetry `InMemorySpanExporter` for in-memory span collection during unit tests. This is shown in multiple test setups, where `InMemorySpanExporter` is used to collect and inspect spans without exporting them externally. You can use this exporter with your tracer provider in your test fixtures or setup code to capture spans in memory for assertions and validation purposes. Example usage can be found in the OpenInference test files, such as: ``` from opentelemetry.sdk.trace.export import InMemorySpanExporter, SimpleSpanProcessor from opentelemetry.sdk.trace import TracerProvider in_memory_span_exporter = InMemorySpanExporter() tracer_provider = TracerProvider() tracer_provider.add_span_processor(SimpleSpanProcessor(in_memory_span_exporter)) ``` According to <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|conftest.py (litellm)>, <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai/tests/openinference/instrumentation/openai/conftest.py|conftest.py (openai)>, and <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai-agents/tests/conftest.py|conftest.py (openai-agents)>. Would you like more detail or a full test example? Sources: - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|conftest.py (litellm)> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai/tests/openinference/instrumentation/openai/conftest.py|conftest.py (openai)> - <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-openai-agents/tests/conftest.py|conftest.py (openai-agents)>
💡 Hint: Mention RunLLM in the thread for followups.
Ben M.
·
RunLLM can you show me what packages I need? opentelemetry.sdk seems to not have those files when just installing arize-phoenix-otel
RunLLM
APP
·
You need to install the `opentelemetry-sdk` package directly, as `arize-phoenix-otel` does not include it as a dependency. The `InMemorySpanExporter` and `SimpleSpanProcessor` are provided by `opentelemetry-sdk`, not by `arize-phoenix-otel` itself. Install with: ``` pip install opentelemetry-sdk ``` This will provide the necessary modules for in-memory span collection in your tests. See <https://arize.com/docs/phoenix/sdk-api-reference/python-packages/arize-phoenix-otel|arize-phoenix-otel docs> and <https://github.com/Arize-ai/openinference/blob/main/python/instrumentation/openinference-instrumentation-litellm/tests/conftest.py|OpenInference test setup>. Would you like a full requirements list for a typical test environment? Sources: - <https://arize.com/docs/phoenix/sdk-api-reference/python-packages/arize-phoenix-otel|arize-phoenix-otel docs>
💡 Hint: Mention RunLLM in the thread for followups.
Ben M.
·
Unfortunately that doesn’t seem to work. I made sure that opentelemetry-sdk was the latest version according to pypi
Xander S.
·
Hey Ben M., it might be worth checking out our test suite for one our auto-instrumentors. https://github.com/Arize-ai/openinference/blob/72a6f06b4ea992a13ba0258ca593a1f7769[…]n-openai/tests/openinference/instrumentation/openai/conftest.py
Xander S.
·
https://github.com/Arize-ai/openinference/blob/72a6f06b4ea992a13ba0258ca593a1f7769[…]on-anthropic/tests/openinference/anthropic/test_instrumentor.py

Ben M.

Thank you, Xander S.! That fixed me up. This is what I ended up with

if config.env != "test":
    tracer = register(
        project_name=config.phoenix_project_name,
        endpoint=f"{config.phoenix_url}/v1/traces",
        headers={"Authorization": f"Bearer {config.phoenix_api_key}"},
        auto_instrument=True,
        verbose=False,
        batch=True,
    ).get_tracer(__name__)
else:
    from opentelemetry.sdk.trace.export.in_memory_span_exporter import (
        InMemorySpanExporter,
    )
    from opentelemetry.sdk.trace.export import SimpleSpanProcessor
    from openinference.instrumentation import TracerProvider

    in_memory_span_exporter = InMemorySpanExporter()
    tracer_provider = TracerProvider()
    tracer_provider.add_span_processor(SimpleSpanProcessor(in_memory_span_exporter))
    tracer = tracer_provider.get_tracer(__name__)

👍1

💡1

RunLLM
APP
·
We've found some places in your existing documentation that contradict the support staff knowledge you've provided! See the docs <https://github.com/Arize-ai/phoenix/blob/main/docs/tracing/how-to-tracing/setup-tracing/custom-spans.md|here> and <https://github.com/Arize-ai/phoenix/blob/main/docs/section-sdk-api-reference/python-packages/arize-phoenix-otel.md|here> Please go <https://app.runllm.com/knowledge/issues|here> to view and resolve all the inconsistencies found for this assistant.
Ben M.
·
Xander S. what if register allowed an in_memory kwarg that would automatically use an InMemorySpanExporter when it is passed True ? I could submit a PR this week if this is something the Arize team is open to
Xander S.
·
Hey Ben M., I would probably avoid that since register is intended for getting up and running in notebooks rather than writing unit tests. In a unit test environment, I would recommend setting up OTel primitives as in the snippet above.
👍1