Glossary · Architecture

Attention Mechanism

The core computational mechanism in transformer models that lets each token in a sequence weigh the relevance of every other token when computing its representation, enabling the model to capture long-range dependencies in text.