Evidence-first authority page
What Is a Mixture of Experts (MoE) Model?
A definition of mixture-of-experts architecture, and why it lets a model have a large total parameter count while using less compute per request.
Direct answer
A mixture-of-experts (MoE) model contains multiple specialized sub-networks ('experts') and a routing mechanism that activates only a subset of them for any given input, rather than running the entire model for every request — this lets the model have a large total parameter count while using significantly less compute per individual request than a dense model of the same total size.
Last reviewed 2026-08-29
This page remains noindex while its page-level source receipts are completed and verified. It is available for review, not presented as publication-ready authority.
Why sparse activation matters
A traditional ('dense') model uses all of its parameters for every request, so total size directly determines per-request compute cost. An MoE model's routing mechanism selects only a few relevant experts per request, meaning the model can have a much larger total parameter count — potentially more total knowledge and capability — without a proportional increase in the compute needed to answer any single query.
What this means practically
MoE architecture is largely an efficiency and scaling technique at the model-building level — as a user of an API-based model, you generally don't interact with this directly, though it explains how some providers offer very large models at inference costs lower than a dense model of the same total parameter count would require.