Tracking Non-LLM Costs for Cumulative Span Analysis

SALIHI A. · 2025-08-07T22:03:28.364Z

Hi everyone, is there a way to track non llm costs and add them to the cumulative cost of the span ? If so, what is the attribute I need to set with the cost ?

Tracking Non-LLM Costs for Cumulative Span Analysis | Arize AI Community

8 comments

· Sorted by Oldest

RunLLM
APP
·
It seems likely that Phoenix currently tracks LLM costs automatically using token counts and model pricing, but there is no documented, built-in attribute for reporting arbitrary non-LLM costs directly on a span. Enhancement requests exist for supporting custom cost reporting on spans for external services, but this is not yet a standard feature or convention in the released platform or semantic conventions documentation. As of now, only LLM-related costs (using token counts and model info) are supported for automatic aggregation and display, and there is no official attribute for manual non-LLM cost injection into the cumulative cost calculation for a span or trace. If you want to track non-LLM costs, you may need to use custom span attributes (e.g., a user-defined attribute like custom.cost) for your own analysis, but these will not be included in Phoenix's built-in cost aggregation or UI displays. For more details or updates on this feature, see the related enhancement discussion and consider following or contributing to the relevant GitHub issue for custom cost support. Would you like a step-by-step breakdown or code example for adding custom attributes to spans? Sources: - <https://github.com/Arize-ai/phoenix/issues/8431|Cost tracking for function calls and external services (enhancement request)>
💡 Hint: Mention RunLLM in the thread for followups.
SALIHI A.
·
🔒[private user] yes, I've looked at the issue and it is exactly what I want, I'll follow it. Thank you for the answer !
Mikyo
·
SALIHI A. we've def. gotten this ask. Curious what cost you are wanting to track? Let us know. Also if you file an issue I think you will find others with this same ask 🙂 It will help us better shape the API. Thanks for the feedback!
SALIHI A.
·
Mikyo I don't think I'll open a new issue since there is already a similar one. I want to track the costs of my agentic application (with adk), most of the costs are LLM costs which are already handled, but sometimes, the tools do stuff that incurs additional costs (image generation for example), and I want to track them in the cumulative cost of the trace. Currently I just add a new attribute cost in the span of the tool, but it isn't taken into account in the total sum.
Mikyo
·
I see I see - SALIHI A. this makes a lot of sense - we currently have a span cost table that we could use - I think the main thing is to keep it clean so you can attribute it for different kinds of operations for later analysis (e.g. tool cost vs llm cost)
SALIHI A.
·
Mikyo yeah, keeping them clearly separated (tool costs and llm costs for example as you said) is what I want, as well as having them aggregated in the cumulative cost. Is it possible to use the span cost table currently ? Here's an example of how I do it for now :
span = tracer.start_span("image_generation") span.set_attribute("cost", 0.245) span.end()
Mikyo
·
Gotcha, no it’s not currently supported so would definitely need to make adjustments. I’ll update the ticket as I go. Appreciate the enhancement idea!
SALIHI A.
·
Thank you Mikyo!
1

RunLLM
APP
·
It seems likely that Phoenix currently tracks LLM costs automatically using token counts and model pricing, but there is no documented, built-in attribute for reporting arbitrary non-LLM costs directly on a span. Enhancement requests exist for supporting custom cost reporting on spans for external services, but this is not yet a standard feature or convention in the released platform or semantic conventions documentation. As of now, only LLM-related costs (using token counts and model info) are supported for automatic aggregation and display, and there is no official attribute for manual non-LLM cost injection into the cumulative cost calculation for a span or trace. If you want to track non-LLM costs, you may need to use custom span attributes (e.g., a user-defined attribute like custom.cost) for your own analysis, but these will not be included in Phoenix's built-in cost aggregation or UI displays. For more details or updates on this feature, see the related enhancement discussion and consider following or contributing to the relevant GitHub issue for custom cost support. Would you like a step-by-step breakdown or code example for adding custom attributes to spans? Sources: - <https://github.com/Arize-ai/phoenix/issues/8431|Cost tracking for function calls and external services (enhancement request)>
💡 Hint: Mention RunLLM in the thread for followups.
SALIHI A.
·
🔒[private user] yes, I've looked at the issue and it is exactly what I want, I'll follow it. Thank you for the answer !
Mikyo
·
SALIHI A. we've def. gotten this ask. Curious what cost you are wanting to track? Let us know. Also if you file an issue I think you will find others with this same ask 🙂 It will help us better shape the API. Thanks for the feedback!
SALIHI A.
·
Mikyo I don't think I'll open a new issue since there is already a similar one. I want to track the costs of my agentic application (with adk), most of the costs are LLM costs which are already handled, but sometimes, the tools do stuff that incurs additional costs (image generation for example), and I want to track them in the cumulative cost of the trace. Currently I just add a new attribute cost in the span of the tool, but it isn't taken into account in the total sum.
Mikyo
·
I see I see - SALIHI A. this makes a lot of sense - we currently have a span cost table that we could use - I think the main thing is to keep it clean so you can attribute it for different kinds of operations for later analysis (e.g. tool cost vs llm cost)
SALIHI A.
·
Mikyo yeah, keeping them clearly separated (tool costs and llm costs for example as you said) is what I want, as well as having them aggregated in the cumulative cost. Is it possible to use the span cost table currently ? Here's an example of how I do it for now :
span = tracer.start_span("image_generation") span.set_attribute("cost", 0.245) span.end()
Mikyo
·
Gotcha, no it’s not currently supported so would definitely need to make adjustments. I’ll update the ticket as I go. Appreciate the enhancement idea!
SALIHI A.
·
Thank you Mikyo!
1