#load-balancing-loss

MoE 구조를 통한 70B급 성능의 14B급 연산 비용 달성

Mixture of Experts (MoE): what it actually does under the hood, and when it pays off