Arturo H.

felix do you know where you read about that? If you have a url or reference I'd love to see it.

felix I had not heard, but it sounds like a good oportunity to talk to them. I am concerned about compatibility with such an approach. But they should just bake it into DSPy.

I am actually trying to make an agent system with event sourcing and cqrs. So that is much simpler to make code changes and just run what you need.

I've noticed that too. I did not look into that.

I don't think so, are you talking about recording the rigth number of tokens?

Ohh yeah. Sorry 😉

The code you need is in this thread

I just added it to a global array, because I was testing.

Ok so this gets you the id? def validate_context_and_answer(example, pred, trace=None): span_index = next(attr['span_id'] for attr in trace[-1] if isinstance(attr, dict) and 'span_id' in attr) You could just add the code to add the evaluation span right there.

It's added at the forward step

    def forward(self, question):
        current_span = trace_api.get_current_span()
        print(current_span.get_span_context().span_id)
        context = self.retrieve(question).passages
        prediction = self.generate_answer(context=context, question=question)
        return dspy.Prediction(context=context, answer=prediction.answer, span_id=current_span.get_span_context().span_id)

That's where data from the result comes back to the framework. Adding the span_id as part of the prediction is a bit of a hack. But you can just ignore it.