Automated essay scoring needs evidence, not just a score.
Modern AI essay grading can read claims, evidence, reasoning, organization, conventions, and source use, but it still needs validation against human raters and rubric anchors.
What this topic means for scoring teams.
What has changed with LLMs
Earlier automated essay scoring systems relied heavily on surface features and engineered signals. LLM-based scoring can inspect meaning, evidence use, reasoning, and rubric language more directly, which makes the system more useful and more accountable.
Where essay scoring fails
Essay grading can break on weak prompts, missing source packets, generic rubrics, copied responses, multilingual writing, adversarial fluency, and score bands where even human raters disagree. Those cases need explicit routing instead of silent automation.
A defensible workflow
The strongest workflow starts with anchor essays, compares AI and human ratings, separates criteria, checks subgroups, and gives teachers a trace that explains why each score was assigned.
Papers behind the topic.
These papers are the anchor shelf for this topic. The Library keeps the citation list short enough to be useful and close enough to assessment work to shape product decisions.
AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition
Yun Wang, Zhaojun Ding, Xuansheng Wu, Siyue Sun, Ninghao Liu, Xiaoming Zhai · AAAI · 2026
Moves automated scoring away from one-shot grading by extracting rubric-relevant components before assigning scores, which is directly relevant to interpretable, audit-ready scoring workflows.
Exploring potential of large language models for automated essay scoring in education
Nimra Mughal, Ali Shariq Imran, Sher Muhammad Daudpota, Zenun Kastrati, Waheed Noor · Discover Artificial Intelligence · 2026
A current open-access AES study comparing GPT and Gemini on benchmark and classroom data, useful for tracking how LLM scoring performs under real rubric and rater-bias conditions.
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
Jiamin Su, Yibo Yan, Fangteng Fu, Han Zhang, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu · Findings of ACL · 2025
Adds a multimodal AES benchmark across lexical, sentence, and discourse traits, highlighting where current MLLMs still lag human evaluation.
Assessing the Reliability and Validity of Large Language Models for Automated Assessment of Student Essays in Higher Education
Andrea Gaggioli, Giuseppe Casaburi, Leonardo Ercolani, Francesco Collova, Pietro Torre, Fabrizio Davide · arXiv · 2025
A useful counterweight to optimistic scoring results: in a real higher-education setting, human-LLM agreement and within-model stability remained weak.
Examining the responsible use of zero-shot AI approaches to scoring essays
Matthew S. Johnson, Mo Zhang · Scientific Reports · 2024
Useful bridge between LLM scoring and assessment responsibility: accuracy is treated as only one part of fairness, explainability, privacy, and accountability.
Applying Large Language Models and Chain-of-Thought for Automatic Scoring
Lee et al. · Computers and Education: Artificial Intelligence · 2024
Focuses on student-written science responses, making it relevant beyond essays and closer to rubric-based constructed response scoring.
Are Large Language Models Good Essay Graders?
Anindita Kundu, Denilson Barbosa · arXiv · 2024
A useful cautionary read: LLM scores can diverge from human raters, especially without careful calibration and review design.
On the Consistency of Automatic Scoring with Large Language Models
Mingfeng Xue, Xingyao Xiao, Yunting Liu, Mark Wilson · Educational and Psychological Measurement · 2026
Directly studies scoring consistency across LLMs, temperatures, and constructed-response datasets, with implications for multi-rater panel design.
Related library topics
Product context
Evalysis connects this research to practical scoring workflows: rubric setup, multimodal intake, judge panels, confidence routing, validation reports, and human review.