Ralph B.

:ty:

we are using the SQLite setup - would moving to PG help with this?

let me check the db setup

we are hosting this ourself

Thank you for your help - cleared things up a lot

:ty: - I had partially not wanted to go with that approach as within pytest it is neither HUMAN or LLM - CODE makes a lot more sense. Under the hood is there anything different between annotations and evaluations?

REST for annotations or evaluations? I have used that successfully for annotations

They are all let through - somewhere it is being re-encoded

If I remove the pytest.mark to enable recording it runs fine