Evaluating Claude 3: A Strong Contender to GPT-4

As we are getting through A LOT of evals on Claude 3. A 2nd Hot Take is .... It is pretty darn good. It's is as close to GPT-4 yet as we have seen yet in the ecosystem. Will release our results shortly. We don't have some of our hardest Evals in a form to publicly release just yet. But we do have tests that show off some gaps where GPT-4 is significantly ahead on the task. That said, nice to see a competitive model hit the market to GPT-4. It's been almost a year since the release with nothing close.