Human-in-the-Loop Grading: Reducing Outliers and Hallucinations

Primary keyword: human in the loop grading

When AI helps grade open-ended work, the goal isn’t to “replace” instructors—it’s to reduce outliers, surface edge cases, and make feedback auditable. Human-in-the-loop (HITL) is the simplest way to do that: put people exactly where model risk is highest, and log what happened so you can prove it later. This aligns with emerging risk frameworks that emphasize human oversight, measurement, and documentation across the AI lifecycle. (NIST Publications , NIST Publications )

Why HITL matters in assessment

LLMs can generate excellent feedback—and occasional false or unsupported statements (hallucinations), especially under distribution shift or ambiguous prompts. Recent work characterizes hallucinations formally and stresses the need for oversight and evaluation loops in high-stakes use. (NIST Publications )

Regulatory and risk guidance points the same way. The NIST AI Risk Management Framework and its Generative AI Profile call for explicit human roles, risk-tiering, and traceable processes. The EU AI Act requires human oversight for high-risk AI and obliges automatic event logging and record-keeping; while not all grading uses fall under “high-risk,” its principles are sensible defaults for campus governance. (NIST Publications , NIST Publications , Artificial Intelligence Act , Artificial Intelligence Act )

Risk map: where errors happen

Use this map to decide where to insert people.

1. Ambiguous rubric fit (judgment risk). Even humans disagree; κ should be monitored. Low κ zones deserve more human review. (PMC )
2. LLM-as-Judge bias. Model evaluators show position/self biases and can be perturbed; never leave them unsupervised on high-impact decisions. (arXiv )
3. Prompt drift & configuration changes. Small template edits or model swaps change behavior; treat prompts/configs as versioned artifacts. (NIST Publications , arXiv )
4. Input quality & structure. Long or multi-modal responses (figures, citations) increase parsing errors; add HITL for those formats. (General best practice; see also risk taxonomies in the NIST GAI profile.) (NIST Publications )

Sampling strategies & thresholds

Goal: review enough work to catch issues early, without re-grading everything.

TL;DR: risk-based sampling beats one-size-fits-all. Many awarding-body policies emphasize representative samples across the full grade spread; some specify numeric heuristics (e.g., 10% with min/max), others warn against fixed rules. (City & Guilds , Safety Training Awards )

What to sample

• Edge cases: borderline bands, large prompt changes, unusually short/long essays, low confidence or high variance between criteria. (NIST GAI profile encourages governance for config changes & pre/post-deployment checks.) (NIST Publications )
• Representative spread: include fails, passes, distinctions to ensure standardization across the range (typical moderation guidance). (City & Guilds )

How much to sample (choose a policy and document it)

Policy A mirrors a published university rule; Policy B is a common heuristic; Policy C is recommended when you track reliability and configuration risk. Avoid only relying on rules of thumb—several bodies advise against fixed percentages; justify your rationale in QA docs. (City & Guilds , Safety Training Awards )

Tip: If any sampled item fails checks (e.g., rubric mismatch or weak rationale), expand the sample (escalation sampling) before release. (Typical moderation escalation guidance.) (solent.ac.uk )

Student appeals workflow (fast, fair, documented)

A clear appeals process protects students and your audit trail. The UK OIA Good Practice Framework recommends transparent steps and timeliness (often aiming to conclude formal complaints or appeals within ~90 calendar days), while many institutional policies set submission windows of ~10–15 working days after results are issued. (oiahe.org.uk )

Suggested 4-stage flow (adapt to your regs):

1. Student request (10–15 working days window). Require references to specific rubric criteria and evidence (paragraphs, citations). (oiahe.org.uk )
2. Triage & acknowledgment (48–72h). Log the request; if it alleges process error or inconsistency, flag the whole cohort for a sample expansion. (Consistent with moderation policies.) (City & Guilds )
3. Independent double-marking. Assign an instructor who did not grade the original, using the same rubric and HITL protocol; reconcile and record outcomes. (Double-marking policies formalize this separation.) (solent.ac.uk )
4. Decision & rationale. Return criterion-level reasons; store artifacts in your evidence inventory for accreditation. (Accreditors expect documented learning assessment processes and evidence.) (MSCHE , AACSB )

Versioning prompts and rubrics (make change traceable)

Treat your rubric, prompt templates, model, and weights like code:

• Version every artifact (e.g., rubric_v4, prompt.evidence.v7, model=gpt-4o-2025-06-xx, temperature, stop conditions).
• Keep model cards (what the configuration is for, known limits) and datasheets for any curated exemplars or datasets used for few-shot prompts. (arXiv )
• Maintain a prompt registry with diffs and rollout plans; many engineering guides advocate version control and safe rollout for prompts/configs. (NIST Publications )
• Pre-flight checks when anything changes (new model, prompt tweak): norming set, κ vs. last version, and a targeted sample expansion. (NIST GAI profile: governance + pre-deployment evaluation.) (NIST Publications )

Evidence logs for accreditation (and regulation)

Most accreditors want documented, reproducible assessment: policies, rubrics, samples, results, and the evidence linking them. Middle States’ Evidence Inventory guidance and “Educational Effectiveness Assessment” standards are explicit about the need to demonstrate student learning with appropriate evidence. Business-school standards (AACSB) likewise look for well-documented Assurance of Learning processes. (MSCHE , AACSB )

If your deployment falls under EU AI Act high-risk categories, you’ll also need automatic event logs and record-keeping (Article 12) and provable human oversight (Article 14). Even outside the Act’s scope, adopting these logging practices strengthens your QA posture. (Artificial Intelligence Act , Artificial Intelligence Act )

Minimum viable log (store per item):

• rubric/version, prompt/template/version, model/version
• criterion-level outputs and rationales
• human actions (reviewer ID, time, decision)
• sampling reason (random/edge case/appeal)
• re-runs (why, when, what changed)

HITL flowchart (from submission to release)

[Student submits work]
        |
        v
[AI criterion scoring] --low confidence/edge rules--> [Flagged]
        |                                              |
      pass                                         enqueue
        |                                              v
        v                                      [Human review]
[Risk-based sampling] --------------------------^     |
        |                                          confirm/adjust
        v                                              |
[Second marker (if disagreement or appeal)] <----------+
        |
        v
[Release results + rationale to student]
        |
        v
[Appeals window] --> [Independent double-mark] --> [Decision & log]
        |
        v
[Retrospective: κ, drift checks, update rubric/prompt versions]

A small “sampling calculator” you can adopt

Pick a policy and write it down in your QA manual.

1. 10% with min/max (Policy A) — “10% of submissions, max 15, min 5; include fails, distinctions, and borderlines.” (Example institutional rule.) (solent.ac.uk )
2. √n rule (Policy B) — simple scaling with cohort size (seen in some internal policies). (Leeds City College )
3. Risk-based (Policy C) — baseline (e.g., 3–5%) plus add-ons for low κ last run, new assessors, new prompts/models, or flagged distributions (advised by moderation/IQA guidance that prioritizes risk over fixed percentages). (City & Guilds , Safety Training Awards )

Don’t forget to publish escalation: if ≥X% of sampled work needs adjustment, expand to Y more and re-check before release. (Moderation expansion is common practice.) (solent.ac.uk )

Metrics to watch (and where HITL helps)

• Cohen’s κ between graders (or grader vs. AI). Track per criterion; aim for “substantial” (≈0.61–0.80) or higher, context-dependent. (PMC )
• Outlier rate: fraction of items where human overrides the model by ≥2 rubric levels.
• Appeal sustain rate: percent of appeals that result in changes; a spike suggests rubric or prompt drift.
• Time-to-resolution: close formal appeals within your policy (many frameworks suggest completing within ~90 days end-to-end). (oiahe.org.uk )

See also: /blog/cohens-kappa-ai-grading and /blog/ai-grading-bias-mitigation.

Putting it together (operational checklist)

1. Define roles: who triages, who double-marks, who approves releases. (Human oversight is not optional in risk guidance.) (NIST Publications , Artificial Intelligence Act )
2. Write your sampling policy (A/B/C) + escalation triggers and publish internally. (City & Guilds )
3. Version rubric, prompts, and models; run a 10–20-item norming set on every change. (NIST Publications )
4. Log everything (inputs, versions, rationales, human actions). If in scope of EU AI Act high-risk, align with Article 12 record-keeping and retention. (Artificial Intelligence Act )
5. Run appeals with clear windows and independent review; close within policy timelines. (oiahe.org.uk )
6. Retrospective after each batch: κ, outliers, root causes; update the rubric/prompt registry.

References & further reading

• NIST AI RMF 1.0 (human oversight, documentation). (NIST Publications )
• NIST Generative AI Profile (govern, evaluate before & after deployment). (NIST Publications )
• EU AI Act—Human oversight (Art. 14) and Record-keeping (Art. 12). (Artificial Intelligence Act )
• Moderation & sampling—Representative spread; avoid naive fixed rules. (City & Guilds )
• Kappa interpretation—use with context. (PMC )
• LLM-as-Judge biases—position/self-bias reminders. (arXiv )
• Model cards & datasheets—document models and data/exemplars. (arXiv )
• Accreditation evidence—MSCHE Evidence Inventory; AACSB AoL. (MSCHE , AACSB )

Want this wired-in? Enable HITL mode in Exam AI Grader: flip on risk-based sampling, route disagreements to a double-marking queue, and export versioned, immutable audit bundles for your accreditors.