felix

also here https://github.com/stanfordnlp/dspy/tree/backend-refactor

https://pypi.org/project/dspy-ai/2.5.0rc3/#history in the Discord they wrote a bit. Seems to start with 2.5. Did not find too much info sadly...

that sounds interesting. have you seen, that dspy want to refactor their backend in an oncoming version?

ok thanks

yes, it always shows 0

did you get the tokens to work?

mate thanks, it wsa working all along with my code. problem was, index returned by px.Client().get_spans_dataframe(project_name="Span-test") is in hex, span id returned with return dspy.Prediction(context=context, answer=prediction.answer, span_id=current_span.get_span_context().span_id) was int. fix: span_id = f"{span_id:x}"

yes 0%| | 0/20 [00:00<?, ?it/s] 9333914778682258366 <class 'int'>

i think i need to set trace = span_id somewhere before? and it seems to be a number here, not a iterateable, but i check that really quick

no sadly not: def validate_context_and_answer(example, pred, trace=None): # Retrieve the current trace # current_trace = trace_api.get_current_trace() span_index = next(attr['span_id'] for attr in trace[-1] if isinstance(attr, dict) and 'span_id' in attr) print(f"Span index: {span_index}") answer_EM = dspy.evaluate.answer_exact_match(example, pred) answer_PM = dspy.evaluate.answer_passage_match(example, pred) return answer_EM and answer_PM results in: 0%| | 0/20 [00:00<?, ?it/s] 345420494385523838 2024-06-18T15:25:28.117772Z [error ] Failed to run or to evaluate example Example({'question': 'At My Window was released by which American singer-songwriter?', 'answer': 'John Townes Van Zandt'}) (input_keys={'question'}) with <function validate_context_and_answer at 0x0000018F27970040> due to . [dspy.teleprompt.bootstrap] filename=bootstrap.py lineno=179 5%|▌ | 1/20 [00:18<05:59, 18.93s/it] where the 345420494385523838 comes from: def forward(self, question): current_span = trace_api.get_current_span() print(current_span.get_span_context().span_id)