Glossary · Infrastructure

Speculative Decoding

An inference optimization where a smaller, faster model drafts several tokens ahead which a larger model then verifies in parallel, speeding up generation when the draft matches the larger model's own predictions.