Why can a model convincingly give an untrue answer?
Answer out loud or in your head first, then compare.
Answer
A model builds an answer from learned patterns and the input it's given. It doesn't automatically check every sentence against an up-to-date database of facts. It can fill missing information with something that fits the text well but isn't true. Such an error is often called a hallucination.
The instruction “don't make things up” helps describe the expectation, but it's not a guarantee. For important information I provide sources, check critical conditions in code and allow the answer “I don't have enough data”. Even giving a link isn't enough: you have to check that the source really supports the claim. A model's good general reputation doesn't replace measurement on our tasks.
Why it matters: The basic limit on using LLMs in a product.
Possible follow-ups: Does a bigger model always solve the problem? When would you use a plain calculation instead of a model?
How did it go? Be honest — “Not yet” puts the question on your review list.
Start with
- F-P01 · What is an LLM? — LLM fundamentals and AI engineering
- F-P03 · What are a token and a context window? — LLM fundamentals and AI engineering
Further reading
- Claude Platform: Tool use overview(opens in a new tab)Docs; accessed 22.10.2026. Client and server tools, the call, and a result matched by call ID.