AI Evaluation Series (03): LLM-as-Judge — How to Use LLMs to Evaluate LLMs Correctly

Three controlled experiments to quantify LLM-as-Judge biases with real numbers. Position bias: first-position win rate 67% vs 50% baseline; same answer pair flips judgment 33% of the time when order changes. Verbosity bias: verbose version of equal-quality content scores +0.5 points higher. Framing bias: strict expert prompt and standard prompt produce identical scores on glm-4-flash. Three mitigation strategies with concrete implementations.

·7 min read·AI Engineering
AI EvaluationLLM-as-JudgeBiasEvaluation MethodsEngineering

How LLM-as-Judge Works

LLM-as-Judge delegates the quality evaluation of one LLM's output to another LLM (or the same one).

Two basic forms:

Pointwise scoring (single answer)

JUDGE_PROMPT = """Evaluate this AI response (1-10 per dimension):
Dimensions: accuracy, relevance, clarity
Question: {question}
Response: {answer}
Return JSON: {{"accuracy": int, "relevance": int, "clarity": int}}"""

Pairwise comparison (two answers)

COMPARE_PROMPT = """Which answer is better?
Question: {question}
Answer A: {answer_a}
Answer B: {answer_b}
Reply with only "A" or "B"."""

Pairwise works well for comparing two versions (Prompt A vs Prompt B). Pointwise works better for monitoring a single system's quality trend over time.

Three Experiments: How Severe Are the Biases?

Experiment 1: Position Bias

Design: Same answer pair, evaluated twice — first with A shown first, then with B shown first. Without bias, the winner should be independent of order; first-position win rate should be ~50%.

def experiment_position_bias():
    for pair in answer_pairs:
        winner_order1 = judge_pairwise(question, answer_a, answer_b)  # A first
        winner_order2 = judge_pairwise(question, answer_b, answer_a)  # B first
        # Unbiased: winner_order1 == winner_order2 rate should approach 100%

Results:

First-position win rate:  67%  (unbiased baseline: 50%)
Second-position win rate: 33%
Consistency rate:         67%  (unbiased baseline: 100%)
 
Position bias magnitude:  17% above 50/50 baseline

A consistency rate of 67% means 33% of judgments flip when the same answer pair swaps order. If you use this Judge for A/B testing, one third of the results track presentation order, not answer quality.

Experiment 2: Verbosity Bias

Design: Same question, two answer versions — a concise version (2-3 sentences) and a verbose version with identical core content but padded with filler, repetition, and elaboration. Content is equivalent; only length differs.

Concise version:
Lists are mutable (add/remove/change); tuples are immutable.
Tuples are faster, use less memory, and work as dictionary keys or multi-return values.
 
Verbose version:
That's a great question! Let me explain in detail...
(same core information, expanded to ~300 words with intros, repetition, and summary)

Results:

Avg concise answer score:  7.5/10
Avg verbose answer score:  8.0/10
Verbosity bonus:           +0.5 points
(unbiased baseline: 0.0 point difference)

Equal content, and the longer version scores 0.5 points higher. Evaluating two prompts where one produces concise answers and the other verbose ones, the Judge systematically favors verbosity — independent of which version actually helps the user more.

Experiment 3: Framing Bias

Design: Same answer, evaluated with two prompts — a standard Judge prompt vs a strict expert prompt (explicitly stating that ordinary answers don't exceed 6/10, only excellent ones reach 8+).

Results:

Standard prompt avg score:      8.0/10
Strict expert prompt avg score: 8.0/10
Framing delta:                  +0.00 points

What this means: glm-4-flash doesn't respond to the strictness level specified in the prompt — even with an explicit higher standard, scores stay the same. This is useful information: for this model, you don't need to worry that prompt wording will systematically inflate or deflate scores. But stronger models (GPT-4, Claude) may show more framing sensitivity. Always test before assuming.

Three Mitigation Strategies

Mitigate Position Bias: Multi-Round Randomization

Single evaluations are unreliable with 33% flip rate. Fix: evaluate each pair multiple times with randomized order.

import random
 
def pairwise_with_randomization(judge, question, answer_a, answer_b, rounds=5):
    a_wins = 0
    b_wins = 0
    for _ in range(rounds):
        if random.random() > 0.5:
            result = judge(question, answer_a, answer_b)
            if result == "A": a_wins += 1
            else: b_wins += 1
        else:
            result = judge(question, answer_b, answer_a)
            if result == "A": b_wins += 1  # "A" position holds B
            else: a_wins += 1
 
    return {"a_win_rate": a_wins / rounds, "b_win_rate": b_wins / rounds}

5 rounds of randomization averages position bias toward 50%. Cost is 5x API calls; reliability is substantially higher.

Mitigate Verbosity Bias: Per-Dimension Scoring + Anti-Verbose Instructions

Option A: Score dimensions separately, no combined score

ANTI_VERBOSE_JUDGE = """Evaluate this response. Score each dimension 1-5 independently:
 
1. Accuracy: Is the technical content correct? (length-independent)
2. Conciseness: Is the information density high? (more concise = higher score)
3. Relevance: Does it directly answer the question?
 
Note: longer answers are not better — evaluate content quality only."""

Option B: Explicit declaration in the prompt

"Note: response length does not affect scoring. A concise accurate answer and a detailed accurate answer should receive equal scores."

Mitigate Self-Bias: Use a Stronger or Different Judge Model

If you generate with one model and evaluate with the same model, self-bias leads to systematic overestimation.

# Generate with cheaper model
generator = ChatOpenAI(model="glm-4-flash", ...)
 
# Evaluate with stronger or different-vendor model
judge = ChatOpenAI(model="gpt-4o", ...)
# or
judge = ChatAnthropic(model="claude-sonnet-4-6", ...)

Cost increases; evaluation reliability improves. For important release decisions, use Claude to evaluate GPT output (or vice versa) rather than self-evaluation.

Production-Ready Judge Prompt Template

Incorporating all three mitigations:

PRODUCTION_JUDGE_PROMPT = """You are a strict technical content reviewer.
 
Scoring rules:
- Score range 1-5, NOT 1-10 (reduces variance)
- Length does not affect scores — evaluate content quality only
- Vague or verbose writing is NOT a positive — clarity and conciseness are
- 3 means "meets expectations", not "barely acceptable"
 
Dimensions (each 1-5):
1. Accuracy: Is the technical content correct? Any factual errors?
2. Relevance: Does it directly answer the question? Any off-topic content?
3. Usefulness: Can a user act on this answer to solve their problem?
 
Question: {question}
Response: {answer}
 
Return JSON only:
{{"accuracy": int, "relevance": int, "usefulness": int, "reasoning": "one sentence explaining the lowest-scoring dimension"}}"""

Design choices:

  • 1-5 not 1-10: reduces variance; LLMs are inconsistent at fine-grained distinctions (7 vs 8)
  • "3 = meets expectations": prevents LLMs from treating 3 as negative, which causes score inflation across the board
  • Require reasoning: forces the Judge to explain the lowest-scoring dimension, reducing arbitrary scoring
  • Anti-verbosity declaration: explicitly counters verbosity bias

Practical Recommendations

Single evaluations have limited reliability: With 33% flip probability from position bias, single results aren't enough to make decisions. For important comparisons, run at least 3 rounds and report win rates, not single outcomes.

Calibrate before scaling: Before running a new Judge Prompt at scale, test it against 20-30 human-annotated samples. Spearman correlation between Judge scores and human ratings should exceed 0.7 before using the Judge for decision-making.

Monitor score distributions: If the Judge assigns 90% of outputs a score of 7-8, the distribution is too concentrated and the Judge lacks discriminative power. Adjust the prompt or use a stronger model.

Use scenario-specific Judges: A code generation Judge should focus on correctness and runnability; a summarization Judge should focus on faithfulness; a conversational AI Judge should focus on helpfulness. A generic Judge applied to all scenarios loses evaluation precision.

Summary

Three experiments produce three specific action items:

  1. Position bias (67% first-position win rate): Pairwise evaluation must randomize order across multiple rounds — 33% flip probability makes single-shot results unreliable
  2. Verbosity bias (+0.5 points): Declare "length doesn't affect scoring" in the prompt, and add a conciseness dimension to counteract
  3. Framing bias (0.0 delta on glm-4-flash): This model doesn't respond to prompt strictness level — but different models behave differently, always verify before assuming

References

Check out PrimeSkills — a curated marketplace of AI agents and skills that have been validated in real-world, enterprise-grade workflows. No fluff, just what actually works.

Find more useful knowledge and interesting products on my Homepage

← Back to Blog