Why do we split a document into chunks?

A whole document can be too long to search and pass to the model conveniently. So we split it into smaller chunks. A well-chosen chunk should keep its meaning, e.g. a whole rule together with its exception, not a random half-sentence.

Example: in a large leave policy, the question concerns one type of leave. It's better to supply the right section than dozens of unrelated pages. Chunks that are too small lose context, and ones that are too large add unnecessary text. Sometimes neighbouring chunks partly overlap so information at the boundary isn't lost. That also increases cost and the number of similar results.