Working with Context

A teacher uploads a 90-page curriculum document and asks the assistant which learning outcomes cover data handling. The answer is confident, neatly formatted — and misses two outcomes buried in an appendix. Was the model “wrong”? Partly. But the teacher could have asked differently: find the sections first, quote them, then answer. Working well with AI is largely about managing what is in front of it, and how you ask it to use that material.

What the model can see

The model itself does not remember your previous chats — any memory comes from a separate product feature, which some assistants switch on by default — and it cannot look into your files or email unless a tool explicitly connects them. In any single conversation it works from four things:

• Its training — general knowledge, fixed at a cutoff date (its knowledge cutoff) and sometimes wrong.
• Hidden instructions from the product maker and from any custom instructions you have set.
• The conversation so far — your messages, its replies, and any files you added.
• Anything fetched during the chat — web search results or connected documents, if the tool has those features turned on.

Everything except its training has to fit in its working space: the context window. Giving the model the right material is the most reliable way to improve its answers — this is grounding in its everyday form.

Tokens and the context window, in plain words

Models read text in pieces called tokens — often a whole short word, sometimes part of a longer one. In English, a token is on average about three-quarters of a word, so 1,000 tokens is roughly 750 words. Watch how a sentence breaks up:

Other languages are often split into more tokens for the same meaning. A 2023 study found large differences between languages in how many tokens the same content needs, with many non-Latin scripts split much more finely than English. In practice, a Bahasa Melayu, Chinese or Tamil document may use up the window faster — and cost more on paid plans — than its English equivalent.

The context window is the maximum number of tokens the model can consider at once, counting your files, the whole conversation and its own replies. Leading models now support windows of hundreds of thousands of tokens, and some advertise a million or more As of 2026 — enough for several long reports. The chat apps do not always give you the full window: free tiers, and some paid plans, get much less, so check your plan. When a conversation outgrows the window, older material is dropped or compressed, and the model may quietly “forget” instructions from the start.

Bringing in files: PDFs, spreadsheets and images

Most assistants accept uploaded files As of 2026. How well they handle them depends on the file.

Strategies for long documents

For anything long, choose a strategy instead of asking one big question. Each has its place.

Instructions that stick: custom instructions, projects and memory

Retyping “I’m a secondary-school science teacher in Malaysia; use British spelling” every time gets old. Assistants offer three kinds of persistence, with different names in different products As of 2026:

• Custom instructions (sometimes called personalisation or preferences): a short standing note about you and how you want answers, applied to every new chat.
• Projects or workspaces: a folder that holds instructions plus reference files, so every chat inside it starts with the same background — say, a style guide and last year’s report.
• Memory: the assistant saves facts from your conversations, or searches past chats, to personalise later answers. Usually it can be viewed, edited or switched off in settings, and most assistants offer a temporary or incognito chat that is not remembered.

Memory is convenient but cuts both ways: it can carry an outdated assumption into a new task, or surface something you mentioned in a personal chat during a work one. Review what is stored now and then, and use a temporary chat for one-off sensitive topics.

Privacy of uploads

Anything you upload or paste leaves your device and is processed by the provider. Depending on the product, plan and settings, it may be stored, reviewed by staff for safety, or used to improve future models. Business and enterprise plans typically promise not to train on your data; consumer plans vary, and the defaults change from time to time As of 2026.

Garbage in, garbage out

The oldest rule in computing still applies. An outdated policy file produces outdated answers; a half-finished spreadsheet produces confident nonsense; notes full of internal abbreviations produce misreadings. A few minutes of preparation pays off:

• Give the latest version, and say which document wins if two disagree.
• Explain abbreviations and internal names the model cannot know.
• Remove irrelevant pages instead of hoping the model will ignore them.
• Say what to do when the material is silent: “If the document does not say, write ‘not stated’.” This one sentence prevents a great many hallucinations.

Key terms on this page

Sources & further reading

• Liu, N. F. et al., “Lost in the Middle: How Language Models Use Long Contexts” (2023; Transactions of the ACL, 2024).
• Petrov, A. et al., “Language Model Tokenizers Introduce Unfairness Between Languages” (2023).
• Anthropic, Prompting best practices — includes long-document and quote-first advice (accessed 2026).
• Google, Prompt design strategies — adding context and breaking prompts into components (accessed 2026).
• OpenAI, Prompt engineering guide — reference text and long inputs (accessed 2026).