Context vs Prompt Engineering: What the Model Actually Sees

You are not writing a prompt. You are filling a window.

Here is the analogy I use with every new teammate: the context window is the model’s desk. It is the only surface the model can see while it works. Your message is a sticky note placed on that desk. Everything else — the system prompt taped to the wall, yesterday’s notes, the documents someone pulled from the archive, the printouts a tool just spat out — is also lying on that desk, and the model reads the whole surface before it writes a word. Anything not on the desk does not exist for it.

This matters because the desk is small, it costs money per square inch, and you — or your code — decide what gets put on it. Getting that right is the difference between an assistant that feels sharp and one that feels drunk.

Here is a question that separates beginners from working AI engineers: when you send a message to a model, what does the model actually see?

Most people picture the chat bubble — the thing they typed. The reality is that the message is a tiny fraction of what goes over the wire. Every API call ships the model a complete context: the system prompt, the conversation history, anything retrieved from memory or a knowledge base, the results of previous tool calls — and somewhere in there, your message.

Prompt engineering is crafting what you say in one message. Context engineering is designing everything else the model sees around it. Prompt engineering is still a skill. But in real products, the prompt is usually the smallest piece.

What counts as context

A typical request to a model includes:

• System prompt — the standing instructions: role, rules, tone, format.
• Conversation history — every earlier turn in this session.
• Memory — facts about the user or project carried over from past sessions.
• Retrieved documents — chunks pulled in by RAG from your knowledge base, matched to the current question.
• Tool results — output from searches, database queries, API calls the model made moments ago.
• The user’s message — often less than five percent of the total.

Notice that most of this is not written by a human at prompt-writing time. It is assembled by code, at runtime, on every single request. That is why it is engineering, not writing.

There is no memory — only this call

The most useful fact about the API is also the least believed: every call is stateless. The model does not remember your previous request. It does not know who you are, what it said a minute ago, or what your product does. Between calls it holds nothing at all.

So how does a chat assistant appear to remember the conversation? Because your code sends the entire conversation again, every single time. Turn five is not “the model continuing” — it is a brand-new request that happens to contain turns one through four plus your new message. Nothing carries over on the model’s side; the illusion of memory is entirely reassembly on yours.

Two consequences follow, and they are the whole discipline:

• There is exactly one artifact per request: the context. The model sees that text, and nothing else. No hidden notes, no database, no memory of last time.
• Anything not in the call did not happen. If a rule, a fact or a constraint was not in that payload, the model cannot use it — and no amount of clever phrasing in the final message will summon it.

This is why “prompt engineering” undersells the job. The prompt is one field in a payload your code assembles. Everything is one call, and you are the one who fills it.

The context window is a budget

The model can only attend to a fixed number of tokens — the context window. A token is roughly three-quarters of a word (a chunk of text the model counts in), and it is the unit for both the window and the bill. Modern models accept 128K, 200K, sometimes a million tokens, but the window is never infinite, and attention is never equally good across it. Treat the window like a budget: every document, every old message, every tool dump spends tokens that could have gone to something more useful.

Here is what one realistic request looks like inside the window. Click through the events — then notice where the user’s actual message lands:

Look at that bar. If you spend your afternoon polishing the 180-token message and leave the other 7,800 tokens to chance, you are decorating the doorknob of a house with no roof.

You do not need a codebase to feel this. Paste this into ChatGPT and watch how little the final question matters compared with what sits above it:

You are helping me test something. Below is a conversation
between a user and an assistant, then a final question.

Answer ONLY the final question, continuing the conversation
naturally.

--- conversation so far ---
User: I run a small bakery. I hate long emails.
Assistant: Noted. Short replies, bakery context from now on.
User: We are out of rye flour until Thursday.
Assistant: Understood - no rye products until Thursday.
--- end ---

Final question: Can I order a rye sourdough for tomorrow?

Answer in one short sentence.

The final question is eight words long and completely ordinary. The answer depends almost entirely on the lines above it. Change one of them — say, “I hate long emails” becomes “I need detailed explanations” — and the same question produces a different answer. That is context engineering: you were not editing the question, you were editing the desk.

Why context engineering is becoming the bigger discipline

Three shifts made context the main event:

1. Apps became multi-step. An agent makes a dozen tool calls per task; each result flows back into the context. The “prompt” is now a stream the code assembles, not a paragraph you type.
2. Models got long windows. With 128K tokens available, the bottleneck moved from “how do I say it” to “what do I put in at all”.
3. Failures became context failures. When a production assistant gives a wrong answer, the cause is rarely the wording of the last message. It is a stale memory, an irrelevant retrieved chunk, a contradictory old instruction still sitting in history.

Prompt engineering is a layer inside context engineering: the message still matters, but it is the last 200 tokens of a design problem spanning thousands.

Where this leaves you

The window is a budget, and most of it is filled by machine. Once you start seeing every request as “what is on the desk right now”, you cannot unsee it — and the questions change from “how do I phrase this” to “what should be here at all”.