LLM judge bias has to be tested before launch.
A judge can look aligned in aggregate while still shifting scores because of response order, writing style, language background, verbosity, or prompt framing.
What this topic means for scoring teams.
Common judge biases
LLM judges can prefer earlier answers, longer answers, fluent answers, familiar styles, or outputs associated with a particular model. In student assessment, the same family of risks can affect multilingual learners, accommodations, handwriting quality, and unconventional solution paths.
Bias testing belongs in the pilot
A pilot should include balanced response order, blinded samples where possible, subgroup agreement tables, score-band analysis, and documented human-review triggers. The output should name where automation is not ready.
Operational controls
Evalysis uses review queues, confidence thresholds, audit traces, and subgroup reporting so bias checks are part of scoring operations rather than a separate slide after the fact.
Papers behind the topic.
These papers are the anchor shelf for this topic. The Library keeps the citation list short enough to be useful and close enough to assessment work to shape product decisions.
Bias and Uncertainty in LLM-as-a-Judge Estimation
James Fiedler · arXiv · 2026
Sharpens the statistics behind judge outputs by showing how corrected estimates can still become unreliable when judge quality or calibration shifts across compared models.
Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations
Manan Gupta, Dhruv Kumar · arXiv · 2026
Looks beneath aggregate agreement by exposing per-document instability, transitivity failures, and criterion-specific reliability differences.
CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges
Haitao Li, Junjie Chen, Qingyao Ai, Zhumin Chu, Yujia Zhou, Qian Dong, Yiqun Liu · ACL · 2025
Gives a concrete inference-time method for reducing option-position and ID-token selection bias in pairwise judge decisions.
Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, Dieuwke Hupkes · GEM · 2025
Shows why simple percent agreement is not enough: judge models can align broadly with humans while still drifting in score scale, leniency, and prompt sensitivity.
A Survey on LLM-as-a-Judge
Jiawei Gu et al. · arXiv · 2024
A broad map of judge reliability, bias mitigation, consistency, and deployment challenges.
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Lianmin Zheng et al. · NeurIPS Datasets and Benchmarks · 2023
Foundational paper for response judging, pairwise preference, position bias, verbosity bias, and human agreement.
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Yang Liu et al. · EMNLP · 2023
Shows how form-filling and rubric-style prompts can improve alignment with human judgments for generated text evaluation.
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Seungone Kim et al. · ICLR · 2024
Important for custom rubrics: open evaluator models can be trained for fine-grained feedback and score criteria.
Large Language Models are not Fair Evaluators
Peiyi Wang et al. · ACL · 2024
A direct warning that judge outputs can be manipulated by response order, requiring balanced position and human-in-loop protocols.
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
Hui Huang et al. · arXiv · 2024
Useful for product design: fine-tuned judges may be strong in-domain yet weaker on generalization, fairness, and aspect-specific evaluation.
Related library topics
Product context
Evalysis connects this research to practical scoring workflows: rubric setup, multimodal intake, judge panels, confidence routing, validation reports, and human review.