Rumors are swirling that GPT-4 may use an advanced technique called Mixture of Experts (MoE) to achieve over 1 tr parameters. This offers an opportunity to demystify MoE