Read Mixtral Before Wednesday for MoE Part 2 Discussion
Pretty decent read on Mixtral before Wednesday for MoE part 2- https://arxiv.org/pdf/2401.04088.pdf
Pretty decent read on Mixtral before Wednesday for MoE part 2- https://arxiv.org/pdf/2401.04088.pdf