I鈥檓 exploring tools to create a cost monitoring dashboard for app, model, and API call costs. I have data logged in the DB via a Python API, including input tokens, output tokens, models used, and subscription keys at the user level. Are there any Arize tools for this, and where can I find documentation on dashboard creation? Preferably those that I can connect the data through pyodbc!
Hi! I can help out. Yes - are these values like input tokens already logged in a DF or like a file? If so, we can consume them and spin up a very simple dashboard with widgets to provide counts, averages and if needed, filtered or groupby. Or if you need tooling to also generate and capture that data, we can use our tracers to capture these details. Relevant documentation - https://docs.arize.com/arize/large-language-models/production-monitoring/token-counting Are you free to hop on a quick call for us to pair on these items?
Hey, Cam! it's already logged in a DF on sql server, would be great to explore a very simple dashboard. Where should I look on that? and is it an open-source as well?
This component would be in the Arize Core platform, app.arize.com for reference; you can upload that data with this SDK https://docs.arize.com/arize/large-language-models/tracing/log-as-inferences If its possible to pass over the head of the DF (or mocked up data for privacy), I can look at it and give you a sample notebook to help you with the ingestion.
