Determine whether your prompt and history exceed a model's maximum context window.
Context overflow occurs when the sum of prompt tokens, conversation history, and reserved completion tokens exceeds the model's maximum context window. When overflow happens, you must truncate history, summarize context, or switch to a model with a larger context window.
Overflow Check
overflow = max(prompt + history + reserved - max_context, 0)
Most APIs return an error. Some frameworks silently truncate the oldest messages. Either way, important context is lost, potentially degrading response quality.
Implement sliding window history, summarize old messages, use RAG to retrieve only relevant context, or upgrade to a model with a larger context window.