Exploring Ada 02 Embeddings: Insights and Reconstruction Findings

Deconstructing Ada 02 Embeddings: I thought this was a pretty impressive write up and hack. In it Linus looks at the information that is contained in Ada 02 embeddings and if he can reconstruct the original data from the embedding. The results are striking and interested in how many people reproduce the results. https://twitter.com/thesephist/status/1698095739899974031?s=20 There seems to be a big idea that maybe many different models learn the same latent space mappings.