The AI model in your chatbot can only process what it is sent and generate a response. It has no memory, no internet access, and no ability to act on anything by itself. Everything else is traditional software built around it. Here is how it all fits together.
flowchart TD
A([You Open the App]) --> B["New Session — Model Knows Nothing"]
B --> D[/You Type a Message/]
D --> E["App Retrieves Full Chat History"]
E --> H["Bundle: Your New Message + ALL Prior Messages"]
H --> I[["AI Model — Receives ONLY This Bundle"]]
I --> J["Model Reads the Whole Conversation From Scratch"]
J --> N["Response Generated Based Solely on What Was Sent"]
N --> O["App Saves Response to Chat History"]
O --> P[/You See the Response/]
P --> Q{Keep Going?}
Q -->|Yes — history grows bigger| D
Q -->|No| R([Session Ends])
style A fill:#c49a3c,color:#0f0e0b,stroke:#c49a3c
style R fill:#c49a3c,color:#0f0e0b,stroke:#c49a3c
style I fill:#6b2f2f,color:#d4cebe,stroke:#8b3535
style J fill:#6b2f2f,color:#d4cebe,stroke:#8b3535
style H fill:#b87a52,color:#0f0e0b,stroke:#b87a52
style E fill:#b87a52,color:#0f0e0b,stroke:#b87a52
Key Takeaways#
- The AI model has no memory. Every message you send is the first time it has ever encountered you, because it reads a fresh transcript of your conversation from the beginning each time
- The chat history you see on screen is stored by the app, not by the model. When you start a new conversation, it is gone from the AI’s perspective
- Features like ChatGPT’s Memory or Claude’s Projects do store information about you, but that is the application doing the work. The model itself still knows only what it is sent in each message
- The model cannot search the web, run code, send emails, or take any action on its own. When an AI chatbot does those things, it is requesting that traditional software do them, and then reading the results. The model only ever reads and writes text
- Token costs compound with every message. Because the entire history is re-sent each turn, you pay for every prior message again with every new one. A five-turn conversation bills roughly six times more tokens than the new content alone would suggest
Most people think of AI chatbots as intelligent systems that remember them, browse the web, and take actions in the world. The reality is more interesting than that, and more limited.
The model at the center of every AI chatbot can only do one thing: process what it is sent and generate a response. Everything else, the memory, the web searches, the file access, the ability to take actions, is traditional software layered around it. Understanding how those layers work changes how you use these tools.
The “Memory” You Think It Has#
ChatGPT has a Memory feature. Claude has Projects. These are real, and they work. But it is the application storing that information, not the model. Here is what actually changes when memory is turned on:
The model still reads everything from scratch. It just gets a sticky note at the top. When a company says their AI “gets to know you,” what they mean is their database does. You can see and delete what has been stored about you in the settings of most apps.
What It Can Actually Do (And What It Cannot)#
The model on its own can do exactly one thing: process what it is sent and generate a response. It cannot search the internet, run code, send email, or take any action in the world. Every capability you see in a chatbot, web search, file reading, booking a meeting, writing code that actually runs, is traditional software bolted on around the model.
Here is what actually happens when you ask a chatbot to look something up:
flowchart TD
A(["You: search for recent AI news"]) --> B["App bundles your message\n+ conversation history"]
B --> C[["AI Model reads the bundle\nand decides to use a tool"]]
C --> D["Tool request:\nsearch_web('recent AI news')"]
D --> E["Traditional software\nruns the actual web search"]
E --> F["Results returned to app"]
F --> G["App adds results to bundle\nand sends it back to the model"]
G --> H[["AI Model reads everything\nincluding results, writes response"]]
H --> I(["You see the answer"])
style C fill:#6b2f2f,color:#d4cebe,stroke:#8b3535
style H fill:#6b2f2f,color:#d4cebe,stroke:#8b3535
style D fill:#2a2218,color:#8a8275,stroke:#3d3830
style E fill:#c49a3c,color:#0f0e0b,stroke:#f0d090,stroke-width:3px
style A fill:#c49a3c,color:#0f0e0b,stroke:#c49a3c
style I fill:#c49a3c,color:#0f0e0b,stroke:#c49a3c
The model never touched the internet. It asked for something, software fetched it, and the results landed back in its context. The model decides what to ask for. The traditional software does the actual work. This is why the same underlying model has completely different capabilities depending on which app you use. The intelligence is in the model. The capability is in the tools someone built around it.
Token Costs Accelerate With Every Message#
AI APIs charge by the token. A token is roughly three-quarters of a word, so a typical message might be 20 to 50 tokens. That sounds cheap. The problem is the re-sending.
Because the entire conversation history travels with every message, you are not paying for one message per turn. You are paying for every message ever sent, plus every response the AI ever wrote, again, with every new turn. The cost does not add up linearly. It compounds: each turn costs the price of all previous turns plus the new one.
Here is what that looks like across a short five-message conversation. Notice that the AI’s responses are just as much a part of the compounding cost as your own messages:
The numbers above are illustrative, but the shape is accurate. Across those five turns, 130 tokens of new content were generated. The total tokens billed: 845. About 85% of what was paid for was history the model had already processed on a previous call.
Scale that to a real enterprise deployment. A support team running hundreds of multi-turn conversations a day, or an AI assistant embedded in a workflow that accumulates context over time, hits this curve hard. The bill is not proportional to the work being done. It is proportional to the square of the conversation length.
There is also a hard ceiling. Every model has a context window limit, a maximum amount of text it can process in one request. Once a conversation gets long enough, the app has to start dropping or summarizing the oldest messages to make room. The model never sees what was trimmed. To it, that part of the conversation never happened. This is why very long chats sometimes feel like the AI has forgotten what you told it early on. It has.
The practical fix is the same whether you are worried about cost or quality: start a new conversation for a new topic. A focused, short context produces better responses and costs less than a long, sprawling one.
What This Actually Changes#
None of this means chatbots are not useful. They clearly are. But understanding the architecture changes how you use them well.
Start fresh conversations often. New topic, new chat. The model will perform better with a focused, relevant context than with a long, sprawling one.
Do not assume it knows you. Even if an app has a memory feature enabled, verify what it has stored. Treat it as a tool with a reference card about you, not a colleague who has been paying attention.
The illusion of continuity is well built. These products are designed to feel like relationships. Knowing that the relationship is actually stateless, rebuilt from a transcript with every message, does not make them less useful. It just makes you a more informed user.
Greymantle Risk Advisory is in the business of making the complex legible. If questions like these come up in your organization and you want someone to help make sense of the technology you are working with, that is what we are here for.