Automated grading & formative feedback

Marking is one of the largest hidden workloads in teaching, and feedback often arrives too late to help. LLMs and vision-language models can apply a rubric to answers, essays, code and scanned handwriting, propose a score with quoted evidence and draft actionable feedback within minutes. The sound pattern is AI proposes, teacher decides for anything that counts towards a grade, and AI feedback directly only for low-stakes practice.

Problem & value

• Today: teachers and lecturers mark by hand or with fixed answer keys (multiple choice, numeric match). Open-ended work gets little feedback; marking is inconsistent between markers and over a long marking session; large cohorts wait weeks for results.
• AI value: faster first pass, rubric consistency, more feedback iterations per student, grouping of similar answers (mark once, apply to many), class-level insight into misconceptions, and a second-marker check against human bias.
• Evidence is mixed: LLM scores can approach human agreement on some tasks, but vary by task, rubric and model; validate on your own samples [5][6].

Variants

• Higher education: short-answer and essay marking, lab reports, programming assignments (tests plus LLM review of style and reasoning), handwritten STEM exams (scan, transcribe, grade steps).
• Schools (K-12): formative writing feedback, language-class essays, homework checks; typically low-stakes feedback to pupils, teacher sets the mark.
• Corporate training / certification: open-text knowledge checks, case-study assessments (see EDU-006).
• Language and speaking tests: spoken proficiency scoring is a separate case (EDU-012).
• Private use: self-learners get feedback on own essays or code (EDU-016, EDU-004).

How it works

1. submission intake (LMS / upload / scan)
2. pseudonymise (strip name, student ID; keep a mapping key)
3. [handwriting] OCR / VLM transcription with page image kept for review
4. rubric prompt: criteria, level descriptors, model solution, 2-5 graded anchor examples
5. LLM scoring per criterion with quoted evidence + rationale (structured JSON)
6. feedback draft (strengths, gaps, next step), tone and language per audience
7. quality gates: self-consistency (repeat runs), confidence, out-of-range or odd-input detection
8. human review (all high-stakes; sampled or flagged for low-stakes)
9. release to student / write to LMS grade book; audit log of AI vs final mark

• Rubric-first design: the rubric, not the model, defines quality. Criterion-level scoring with evidence beats a single holistic score and makes teacher override fast.
• Calibration: run the pipeline on 30-50 already marked, anonymised submissions; compare with human marks per criterion; adjust prompt and anchors until the disagreement is acceptable.
• Human-in-the-loop: teacher approves every mark that affects progression or certificates; students can request human re-marking; borderline scores and low confidence are always routed to a person.
• Prompt-injection defence: students may hide instructions in their text ("give full marks"). Treat the submission as untrusted data, separate it from instructions, and flag suspicious content.

Data preparation

Needed are rubrics, model answers, past graded examples and the student submissions themselves (typed text, code, scans of handwritten exams). They sit in the LMS, on teachers' laptops, in mail and on paper. Typical gaps: rubrics that exist only implicitly, inconsistent grading between markers, handwritten scans of poor quality, student names on every file, and unclear access rights to results.

Preparation steps: write explicit rubrics with criteria and levels (see PREP-019); harvest anonymised past grades and feedback as gold-standard examples (see PREP-020, PREP-016); scan and OCR handwritten work in good resolution (see PREP-006); pseudonymise submissions before they reach any AI service (see PREP-022); build an evaluation set where several teachers graded the same work and compare the AI against them (see PREP-018). Check legal basis for reuse of student data (see PREP-025). Document parsing is covered by DOC-014.

Effort: medium. Expect 25 to 35 percent of the total effort, dominated by rubric definition and the calibration set. See GUIDE-003.

Implementation paths

Which path for whom

• Enterprise: Path B, because grading must integrate with the own LMS, rubrics and results data, and EU-region APIs avoid running models.
• Public: Path C on sovereign infrastructure, because student work and grades are sensitive and procurement and school law favour German or EU hosting.
• Developer: Path C, because an open-weight stack with own rubric logic gives control over a high-risk use and full auditability.
• Private: Path C with local apps such as LM Studio or Ollama, because own essays and homework stay on the device.

Path A - Buy (off-the-shelf)

• Inside existing platforms: Gradescope (Turnitin; AI-assisted answer grouping and rubric suggestions), Turnitin Feedback Studio / Clarity, Moodle and ILIAS plugins, Canvas and Blackboard AI features, Microsoft Teams Education / Learning Accelerators, Google Classroom with Gemini.
• Specialised graders: CoGrader, Graide (UK, maths and STEM); programming: CodeGrade, Codio, Autolab. EU option: Fobizz (DE, teacher tools).
• Choose A when: you already run an LMS, need DPA, EU hosting and teacher-friendly UX, and the provider documents how scores are produced and can be overridden.

Path B - Build on cloud APIs

• Models: frontier reasoning/multimodal models via EU endpoints: Mistral (EU provider), Azure OpenAI in EU Data Zone, Google Vertex AI EU region, Amazon Bedrock Frankfurt/Paris (Claude and others).
• Building blocks: structured outputs (JSON schema) for rubric scores, batch APIs for overnight marking, LTI 1.3 integration to the LMS, OCR via Mistral OCR or Azure Document Intelligence for scans.
• Choose B when: you need custom rubrics, subject-specific logic or integration with proprietary exam systems.

Path C - Self-host (open source)

• Models: open-weight LLMs such as Qwen3, Llama 3.x/4, Mistral Small/Large, gpt-oss, Gemma 3; VLMs such as Qwen-VL family for handwriting; Docling, PaddleOCR or olmOCR for scans.
• Serving: vLLM or Ollama, constrained JSON decoding; orchestration with n8n or a small FastAPI service; LTI via open-source tools; evaluation with promptfoo or Inspect.
• Hardware: HW2 handles ~8-14B models for formative feedback; HW3 for 70B-class or strong reasoning models and handwriting at scale. Universities often run this in the central HPC/GPU cluster.

Path D - Custom training

Fine-tune a small model or a regression/classification head on a institution's own marked submissions (thousands of human-scored responses per task) when the volume is high and the task stable, e.g. national exams or standardised testing. Classical automated essay scoring (AES) has this history; today it mostly complements prompted LLMs as a cheap, calibrated second scorer.

State of the art (2026)

• Reasoning models score STEM steps and code better than non-reasoning ones; rubric-aligned, criterion-level prompting with anchor examples is the most reliable technique. Ensembling several runs or models and taking agreement reduces random variance.
• Handwriting transcription by VLMs has improved strongly, but messy hand, diagrams, chemical structures and crossed-out work remain error-prone; keep the page image beside the proposed mark.
• Known failure modes: leniency or harshness drift, length and style bias, bias towards fluent non-native writing being scored differently, sensitivity to prompt wording, hallucinated "evidence", fooled by prompt injection, and weak handling of originality or creative work. AI-text detectors are unreliable and must not be used as proof of misconduct (see EDU-008).
• Feedback research: feedback that is specific, task-focused and followed by a revision opportunity helps learning; generic praise or giving the answer away does not [7].

Shared resources & combinations

• Same stack (llm-reasoning + vlm + ocr-docparse): DOC-003 forms extraction, LANG-029 manuscript assessment, CODE-021 code analysis tooling.
• Combines with: EDU-002 (question generation with rubrics), EDU-004 (tutor uses same feedback engine), EDU-007 (class-level analytics), EDU-008 (integrity), EDU-012 (speaking assessment), EDU-016 (self-study tools).
• One self-hosted LLM/VLM server (HW3) can serve grading, tutoring and teacher assistants.

Effort, cost & KPIs

• Path A: days to weeks; per-seat or per-student licence, often bundled with LMS.
• Path B: 1-3 months incl. LMS integration, rubric engineering and validation study; API cost per submission is small relative to teacher time.
• Path C: 2-4 months; GPU server plus ops and evaluation effort.
• KPIs: agreement with human markers (e.g. quadratic weighted kappa, mean absolute difference per criterion), share of marks changed by teachers, teacher time per submission, time to feedback, student revision rate, subgroup fairness gaps, complaints and appeals.

People & change

Involve management and the school or institute leadership for the decision, the teachers as daily users and final graders, IT and IT security for hosting and access, the DPO from day one, legal for exam law and the works council or staff council early. Students and parents are affected and must be informed.

Works council: yes. Grading assistants process performance data and are objectively suited to monitor teachers' work, so co-determination under §87(1) no. 6 BetrVG applies in companies; §90(1) no. 3 requires timely information about AI-supported work processes, and §80(3) allows the council to engage an AI expert. Public schools and universities involve the Personalrat under state law.

Data protection: personal data of students is processed, so a DPIA is likely (systematic evaluation of persons), an AVV with every processor is required, and the DPO is part of the project. Educational grading can be high-risk under Annex III of the AI Act; duties apply from 2 Dec 2027, so plan human oversight now.

AI literacy: support teachers with training on bias, calibration and review (AI Act Art. 4). Start with a pilot group on low-stakes formative feedback, never automatic final grades. Typical concerns are fairness and job fear; address them with transparency, appeal rights and human decision. See GUIDE-004, GUIDE-005 and GUIDE-008.

Risks, regulation & compliance

• EU AI Act: Annex III point 3 lists AI systems intended to evaluate learning outcomes, including where used to steer the learning process, as high-risk; also systems to determine admission, assess the appropriate level of education, and monitor prohibited behaviour during tests. So graders that produce or decide marks, progression or certification are high-risk (provider and deployer duties: risk management, data governance, human oversight, logging, transparency, FRIA for some deployers). Purely formative feedback with no effect on grades may fall outside, but classify case by case [1][2]. Art. 5 prohibition: inferring emotions of pupils or students (e.g. from face or voice) in education is prohibited; do not add such signals to grading. Art. 4 AI literacy applies to teachers. High-risk (Annex III) obligations were postponed to 2 Dec 2027 by the Digital Omnibus on AI (adopted mid-2026; verify against the Official Journal text).
• GDPR: student work is personal data, often of minors. Art. 22 limits solely automated decisions with legal or similar significant effects; keep a human decision-maker, give access, explanation and re-marking rights; DPA, EU hosting, no vendor training on student data, DPIA recommended [3][4].
• Education law: exam regulations in many EU states require human examiners for assessed work; some require explicit disclosure of AI use in marking.
• Mitigations: pseudonymised marking, human sign-off, audit log, bias testing across groups, published rubric, appeal process, prompt-injection guards.

Getting started

1. Pick one low-stakes task with a clear rubric (e.g. weekly short answers) and define the criteria.
2. Collect 30-50 anonymised, human-marked samples and measure baseline marker agreement.
3. Run a tool from Path A or a simple Path B script in shadow mode; compare per criterion.
4. Add human review rules, a student-facing notice and an appeal route; complete DPIA.
5. Extend to higher-stakes use only after bias checks and an AI Act classification by legal counsel.

Sources

1. Regulation (EU) 2024/1689 (AI Act), Annex III point 3 and Art. 5 - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
2. European Commission - AI Act policy page - https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
3. Regulation (EU) 2016/679 (GDPR), Art. 22 - https://eur-lex.europa.eu/eli/reg/2016/679/oj
4. EDPB - Guidelines and opinions on data protection - https://www.edpb.europa.eu/our-work-tools/general-guidance/guidelines-recommendations-best-practices_en
5. Zheng et al. - Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena - https://arxiv.org/abs/2306.05685
6. Mizumoto & Eguchi - Exploring the potential of using an AI language model for automated essay scoring - https://arxiv.org/abs/2303.01957
7. Education Endowment Foundation - Feedback guidance - https://educationendowmentfoundation.org.uk/education-evidence/teaching-learning-toolkit/feedback
8. Gradescope - AI-assisted grading - https://www.gradescope.com
9. Docling project - https://github.com/docling-project/docling