Practice · Fast-moving · Advanced
LLM Evaluation
Measuring the quality, safety and cost of language-model outputs where there is rarely one correct answer.
What LLM Evaluation is
Evaluation combines automatic checks, model-as-judge scoring against a rubric, and human review on a sampled set, run against a fixed suite of representative cases.
How it works
Teams build golden datasets from real traffic, define rubrics, score with deterministic assertions where possible, and track pass rates over time. Judges themselves must be validated against human agreement.
Why it matters
It is the only way to make prompt and model changes safely, and the weakest area in most AI teams.
Common uses
- →Prompt and model change gates
- →Safety and refusal testing
- →RAG groundedness checks
- →Agent trajectory review
Strengths
- ✓Turns subjective quality into a tracked metric
Watch for
- ✓Judge models carry their own biases
- ✓Golden sets go stale
Continue exploring
More in this collection
Browse all AI Concepts