Glossary · Infrastructure

Prompt Cache

A mechanism that reuses a model's processed representation of a repeated prompt prefix across requests, reducing latency and cost for applications that send the same system prompt or context repeatedly.