Glossary · Architecture

Sliding Window Attention

An attention mechanism variant that restricts each token's attention to a fixed nearby window rather than the full sequence, reducing compute cost for very long contexts.