How do you estimate the load of an assistant for 1,500 employees?

The number of accounts doesn't yet tell you how much work arrives at once. I ask how many people use it at the same time, how often, how long the tasks are and how many model calls one task makes. I consider large document imports and short interactive questions separately.

If one task takes long, even a moderate inflow can mean a lot of work in progress. The constraint may be the database, the network or the model's limit, not just CPU. I call the numbers I assume during the interview assumptions and show how I'd check them by measurement. I also design the behaviour when a backlog builds up, instead of assuming auto-scaling handles any load.