How do you choose a model and assess the cost of a whole task?
Answer out loud or in your head first, then compare.
Answer
I compare the available models on examples similar to users' tasks. I look at correctness, time, refusals and cost. A cheaper single call can make a more expensive task if it needs many retries or a lot of manual correction.
When calculating cost, I include the whole path: input, output, extra model steps, retrieval, tools and attempts after errors. I also set a work limit for one task. I don't simply pick the winner of a public leaderboard, because our documents and permissions are a different problem. If I route different tasks to different models, I check that this decision really improves results and doesn't break security.
Why it matters: The role requires ownership of working, cost-effective products.
Possible follow-ups: What matters more in a given process: cost, time or the cost of a mistake?
How did it go? Be honest — “Not yet” puts the question on your review list.
Start with
- F-P01 · What is an LLM? — LLM fundamentals and AI engineering
- F-P03 · What are a token and a context window? — LLM fundamentals and AI engineering
Further reading
- Microsoft Foundry: Observability in generative AI(opens in a new tab)Docs; accessed 22.10.2026. Evaluation, monitoring and tracing across the solution's lifecycle.