What does LLM-as-a-Judge mean?
Review note: The answer refers to the original author's CV or target job offer — adapt it to your own experience.
Answer out loud or in your head first, then compare.
Answer
It's using a model to judge the output of another call, e.g. whether a summary covers the key facts. The model gets the evaluation criteria and the material to compare the answer against. It can speed up evaluating a large number of examples.
Such a judge can be wrong too. It may prefer long answers, or treat a nice justification as proof of correctness. So I compare its scores with human scores and use code for clear-cut conditions, e.g. amounts or permissions. A score of “9/10” doesn't automatically mean a 90% probability of truth. First you have to establish what that scale actually measures.
Why it matters: The term is in your CV; it's worth being able to say it in plain language.
Possible follow-ups: How will you check whether the judging model is reliable?
How did it go? Be honest — “Not yet” puts the question on your review list.
Further reading
- Claude Platform: Define success criteria and build evaluations(opens in a new tab)Docs; accessed 22.10.2026. Success criteria, representative data and kinds of graders.