How do you choose the size and boundaries of document chunks?
Answer out loud or in your head first, then compare.
Answer
I look at the content and users' questions. A chunk should contain enough context to be understood. I don't want to cut an exception off from its rule, or a header from its table. But a chunk that's too large can bury the right information in unnecessary text.
I can compare several splits on the same questions and check whether the needed chunk is found. Partial overlap between neighbouring chunks helps at the boundaries, but it duplicates data and increases cost. I keep the link to the document, page and version. Sometimes I search a short chunk but attach the larger section it belongs to for the answer.
Why it matters: A practical choice, not one magic number of characters.
Possible follow-ups: How do you treat code, tables and scans compared with plain text?
How did it go? Be honest — “Not yet” puts the question on your review list.
Start with
- G-P01 · What is RAG? — RAG and retrieval
- G-P02 · What are embeddings? — RAG and retrieval
Further reading
- Azure AI Search: Chunk large documents(opens in a new tab)Docs; accessed 22.10.2026. Choosing chunk boundaries and keeping context.