What an AI assistant really is
How it works “behind the scenes”, in plain words. And why it is sometimes wrong with great confidence.
You’ve surely heard of ChatGPT, Claude, Gemini, Mistral or Copilot. They are all AI assistantA program you “chat” with by typing or speaking: ChatGPT, Claude, Gemini, Mistral, Copilot and similar. It replies by generating text (and sometimes images or code).: programs you can “chat” with as you would with a person. You type a question, you get an answer.
The names and the button colours change, but they all work the same way. That’s why this course isn’t about any particular product: what you learn here works for all of them.
A very good “sentence finisher”
Inside every assistant there is a Language model (LLM)The “engine” inside the assistant. A program that has read huge amounts of text and learned to predict which word comes next.. Think of it as your phone keyboard’s autocomplete… but enormously more powerful.
During TrainingThe phase in which the model learns by reading text. It doesn’t learn from you during the chat, unless the service uses conversations to improve (check the settings!)., the model read a gigantic amount of text (books, websites, articles). From that it learned which words tend to follow others. When you write to it, it doesn’t look the answer up in an archive: it builds it by choosing the most suitable word, then the next one, then another.
Did you notice? At low temperature you almost always get the same, “most obvious” sentence. At high temperature it gets more varied, sometimes funny. Assistants use something in between: that’s why if you ask the same question twice, you may get two different answers.
The most important idea of this lesson
An AI assistant doesn’t “know” things the way a person does. It produces the most likely text for your request. Usually likely text is also correct… but not always.
What it’s good at
Precisely because it has read so much, an AI assistant is excellent with words and ideas:
- Writing and rewriting: emails, messages, letters, posts. Changing the tone too (kinder, more formal, shorter).
- Summarising: a long document, an article, meeting notes.
- Explaining: a hard concept, a technical term, a bill, with examples tailored to you.
- Brainstorming: ideas for a gift, a name, a trip, a dinner.
- Translating and fixing grammar mistakes.
- Organising: turning messy notes into a list, a plan, a table.
Many assistants today are also MultimodalAn assistant that understands not just text, but also photos, documents, audio or voice.: you can send them a photo, a PDF, or talk to them.
What it’s bad at (and why)
1. It makes things up: “hallucinations”
Since it produces likely text, it sometimes writes things that are false but believable: a wrong date, a quote nobody said, a book that doesn’t exist, a link that goes nowhere. This is called a HallucinationWhen the AI writes something false in a confident tone: a made-up number, a quote that never existed, a broken link.. And the tone stays confident and convincing even when it’s wrong.
2. It has an expiry date
The model stopped studying at some point: that’s its Knowledge cutoffThe point when the model stopped “studying”. It knows nothing about later events unless it can search the web.. It knows nothing about what happened afterwards, unless the assistant can search the web (many now do: you can usually tell because it shows sources).
3. It has limited memory
During a chat, the assistant remembers what you’ve said… but only up to a point. This memory is called the Context windowThe chat’s “short-term memory”: how much text the AI can keep in mind in one conversation. In a very long chat, the oldest parts may be forgotten.. In very long conversations, the oldest parts can “drop out” of memory.
See? The last answer suggests meat: “I’m vegetarian” had dropped out of memory.
The example is short on purpose: here the memory only holds 5 messages, so you can see the effect in a few clicks. In reality the context window is much bigger: today’s assistants can keep hundreds of pages of text in mind, so the problem only shows up in really long conversations or when you upload very large documents.
Practical tip: when you change topic, open a new chat. And if a conversation is very long, repeat the important information.
What about different chats?
Usually, every new chat starts from scratch. Some assistants have a “memory” feature that remembers a few things between chats: you’ll find it in the settings, where you can view, edit or turn it off.
4. It doesn’t count, calculate or “see” like we do
It sounds odd, but a language model can get a simple sum or the number of letters in a word wrong. That’s because it reads text in small pieces called TokenA small piece of text (a word, or part of one). The AI reads and writes one token at a time., not letter by letter. For important calculations, ask it to use a calculation tool (if it has one) or check with a calculator yourself.
Summing up
| The AI assistant… | So you… |
|---|---|
| writes the most likely text | check important facts |
| can make things up confidently | don’t trust the tone alone |
| has a knowledge cutoff | ask for sources on recent news |
| has limited memory | open new chats and repeat key info |
| is great with words and ideas | use it to write, explain, organise |