Quiz & exam question generation

Writing good test items is slow and skilled work. LLMs read the course material and draft questions, answer keys, plausible distractors and explanations in minutes, mapped to learning objectives. Humans review, edit and approve. Used for low-stakes practice and formative quizzes the gain is large; for graded exams the AI draft is only the start of a validated item-development process.

Problem & value

• Today: teachers, lecturers and corporate trainers write items by hand, often at night before a deadline. Item banks are small, get leaked or memorised, and weak distractors or ambiguous stems lower the test's validity. Compliance trainings reuse the same ten questions for years.
• AI value: many candidate items per source document, variation (parallel forms, many numeric variants), coverage checks against objectives, quick quizzes for learners from their own notes, and multilingual versions. Typical result: authoring time shifts from writing to reviewing.
• Evidence: published studies on LLM-generated multiple-choice questions generally find a large share usable with little or no editing, but quality varies by subject and prompt, and distractor quality and higher-order (application/analysis) items are the weak points. No universal accuracy figure is reliable; measure on your own material.

Variants

• School teacher: quick formative quizzes aligned to a curriculum; see EDU-009.
• Higher education: large item banks for LMS (Moodle, ILIAS, OLAT, Canvas), parametrised STACK/numeric questions, case-based and open exam questions with marking schemes.
• Corporate / compliance training: knowledge checks after microlearning (EDU-006), certification question pools, awareness quizzes.
• Professional certification bodies: formal item-development with psychometrics; AI as drafting aid.
• Private use: self-made quizzes and flashcards from own notes, see EDU-016.

How it works

1. source material (PDF/slides/video transcript) + learning objectives
2. parsing & chunking (OCR/layout, transcript)
3. item generation (LLM, schema-constrained: stem, key, distractors, rationale, source quote)
4. automatic checks (key verifiable in source, single correct answer, no clues, duplicates, reading level)
5. human review (subject expert accepts / edits / rejects)
6. item bank with metadata
7. export (QTI / Moodle XML / H5P / LMS API)
8. pilot use
9. item statistics (difficulty, discrimination)
10. revise or retire

• Grounding: retrieval over the supplied material, with every item citing its source passage, keeps questions on-syllabus and reduces invented facts.
• Item types: MCQ, multiple response, true/false, cloze, matching, short answer with rubric, numeric with parameters (computed by code, not by the LLM), scenario-based questions.
• Second-model check: a separate "solver" model answers the item blind; disagreement with the key flags flawed items. A reasoning model is useful for maths and logic items.
• Human-in-the-loop: mandatory expert approval before any item enters a graded assessment; post-use statistics from real cohorts decide whether an item stays.

Data preparation

Quiz generation needs the course material the questions must be based on: scripts, slides, textbooks, standards and learning objectives. In an SMB these live on file shares, in the LMS, in mail attachments and on paper handouts. Typical gaps: several conflicting versions, scans without text, learning objectives not written down, no existing item bank and no record of which questions performed well.

Preparation steps: define the authoritative source set per course (see PREP-013); OCR scans (see PREP-006); write down learning objectives and a simple category scheme such as topic and difficulty (see PREP-019); harvest earlier exam questions and answer statistics as examples and quality labels (see PREP-020); build a small test set of good and bad items with subject experts (see PREP-018). Keep secret exam items separate from material that goes into external tools. Parse documents with DOC-014.

Effort: low to moderate. Expect about 15 percent of the total effort, mostly for source curation and the first expert-reviewed sample. Use GUIDE-003 as a checklist.

Implementation paths

Which path for whom

• SMB: Path A, because quiz generators in LMS and authoring tools are mature, EU-hostable and cheap for standard training content.
• Enterprise: Path A for ordinary training quizzes as a commodity function; move to B when exams are certification-relevant and need item banks and psychometrics.
• Developer: Path B, because the value lies in integrating generation, review workflow and item bank into an own product.
• Public: Path B on an EU or German cloud, because exam content and later results need sovereign hosting and defensible processes.
• Private: Path A, because consumer apps generate practice questions from own notes without any setup.

Path A — Buy (off-the-shelf)

• LMS-native: Moodle (AI subsystem with providers, plus AI question-generation plugins), Canvas, Blackboard, Microsoft Teams/Forms and Google Forms/Classroom AI features; check EU tenant and data use.
• Quiz/teaching tools: Kahoot!, Quizlet, Wooclap (EU, Belgium), Quizizz, Conker and similar generators; for German schools fobizz and telli (provider-hosted, school-oriented tools).
• Corporate: authoring tools such as Articulate, Easygenerator (EU, NL), 360Learning (EU, FR), Docebo.
• Choose A when: low-stakes quizzes, small team, LMS already in place.

Path B — Build on cloud APIs

• Models: frontier LLMs (Claude, GPT, Gemini) or Mistral (EU) with structured outputs / JSON schema; EU regions through Azure OpenAI, Google Vertex AI or AWS Bedrock; Mistral and Aleph Alpha-style EU providers for data-residency requirements.
• Stack: document parsing (Azure Document Intelligence, Mistral OCR), embeddings + vector DB for grounding, workflow in n8n or Python, export via QTI 3.0 or Moodle XML/GIFT, LMS web-service APIs.
• Choose B when: you need an item workflow with review queue, metadata and house style across an institution.

Path C — Self-host (open source)

• Models: open-weight LLMs such as Qwen3, Llama, Mistral/Mixtral, Gemma or gpt-oss, served with vLLM or Ollama; Docling or MinerU for parsing; Qdrant/pgvector for retrieval.
• Tools: H5P (open source) and Moodle plugins, STACK for parametrised maths questions, Open WebUI as front end.
• Hardware: HW2 for ~14-32B quantised models, enough for item drafting; HW3 for stronger reasoning models.
• Choose C when: exam material is confidential and must not leave the institution, or student data is involved.

Path D — Custom training

Rarely needed. Fine-tuning on an institution's own validated items (with statistics) can improve house style and difficulty calibration; difficulty-prediction models need thousands of items with response data.

State of the art (2026)

• Structured outputs, long-context reading of whole scripts and reasoning models make grounded MCQ generation with source citation routine; blind-solver verification is a common quality gate.
• Difficulty and discrimination prediction from text alone is still unreliable; real response data (item response theory, classical test statistics) remains the standard.
• Weak spots: implausible or too-obvious distractors, several defensible answers, questions answerable without the material, clues in the stem, shallow recall-level bias, wrong numeric results, cultural or language bias, and near-duplicates of public or internet items (leakage into model training data).
• Interoperability: 1EdTech QTI 3.0 and Moodle XML/GIFT are the practical exchange formats [3].
• AI-written items are increasingly used for practice; fully automatic high-stakes assessment is not accepted practice.

Shared resources & combinations

• Same stack (llm-general, document parsing, embeddings/vector DB): EDU-001 course material generation, EDU-006 microlearning, EDU-009 teacher assistant, EDU-016 study tools, DOC-014 document parsing for RAG.
• Downstream: EDU-003 automated grading and feedback uses the rubrics and keys created here; EDU-008 integrity measures; EDU-007 analytics consume item statistics.
• One self-hosted LLM server (HW2/HW3) can serve all these education use cases.

Effort, cost & KPIs

• Path A: hours to days; mostly licence cost, often bundled in LMS or quiz-tool subscriptions.
• Path B: 2-6 weeks for pipeline, review UI and LMS export; API cost per item is small compared with expert time.
• Path C: 1-3 months incl. hosting; one GPU server and operations.
• KPIs: acceptance rate without edits, expert review time per accepted item, share of items with flawed keys found in review, item difficulty/discrimination after pilot, coverage of learning objectives, learner feedback on clarity, reduction in authoring hours.

People & change

Involve the department lead and the subject experts who sign off every item, the trainers as daily users, IT for LMS integration and the DPO when learner data or results are connected. Legal or compliance matters for certification-relevant exams, HR when exam results feed into appraisals.

Works council: depends. Pure item generation does not monitor anyone, so usually no co-determination. As soon as results or logs are linked to named employees, the system is objectively suited to monitor performance and §87(1) no. 6 BetrVG applies; the council must in any case be informed about AI in work processes (§90(1) no. 3 BetrVG). Schools and authorities involve the staff council (Personalrat).

Data protection: generating items from course texts needs no personal data; an AVV with the vendor is still required. A DPIA is likely if the tool is combined with evaluation of persons at scale.

AI literacy: support staff with training on item quality, bias and answer-key checks (AI Act Art. 4). Start with one course as pilot, let experts review all items, and state clearly that humans decide on exam content and grading. Typical concern: fair, valid exams; address it with review and pilot statistics. See GUIDE-004, GUIDE-006 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: drafting practice quizzes or items that teachers review is typically minimal risk. Annex III point 3 lists as high-risk systems used to evaluate learning outcomes, to determine admission or access, to assess the appropriate level of education, or to monitor prohibited behaviour during tests [1]. If generated items are used in an automated way to steer a student's learning path or decide on admission, progression or grades, assess high-risk status; a human-reviewed item bank used in a human-graded exam is usually outside it. Art. 4 AI literacy applies to staff; Art. 50 transparency applies where learners interact directly with a generator. Art. 5 prohibits emotion recognition in education (relevant to proctoring, not to item generation). Check the current application dates of the high-risk rules.
• GDPR: source material rarely contains personal data, but uploads of student work, learner profiles used to personalise quizzes, or quiz results are personal data (Art. 6, DPA with provider, EU hosting, DPIA for minors) [2]. Pupils' data in schools is subject to state-level school laws in Germany.
• Copyright: generating items from licensed textbooks needs rights to process the content; check publisher terms and the text-and-data-mining exception (Art. 4 DSM Directive) and opt-outs.
• Quality and fairness: wrong keys harm learners and invite appeals; bias in wording; exam validity and legal challengeability (Prüfungsrecht) require documented human responsibility.
• Mitigations: source-grounded items, blind-solver check, mandatory expert sign-off, piloting with statistics, item-bank access control, versioning and audit trail of AI vs. final item.

Getting started

1. Pick one course; collect its learning objectives and 3-5 source documents.
2. Try the LMS/quiz tool you already license (Path A) on low-stakes quizzes; keep a log of faults found.
3. Write a house item-writing guide (stem rules, distractor rules, Bloom levels) and put it into the prompt.
4. Add review workflow: source quote per item, blind-solver check, expert accept/reject.
5. Pilot with one cohort, compute difficulty and discrimination, retire weak items; then scale.

Sources

1. Regulation (EU) 2024/1689 (AI Act), Annex III point 3 and Arts. 4, 5, 50 - https://eur-lex.europa.eu/eli/reg/2024/1689/oj
2. Regulation (EU) 2016/679 (GDPR) - https://eur-lex.europa.eu/eli/reg/2016/679/oj
3. 1EdTech - QTI 3.0 assessment interoperability standard - https://www.1edtech.org/standards/qti/index
4. Moodle Docs - AI subsystem and AI providers/placements - https://docs.moodle.org/en/AI_subsystem
5. H5P - open-source interactive content - https://h5p.org/
6. STACK - open-source maths assessment system - https://stack-assessment.org/
7. Docling project - https://github.com/docling-project/docling
8. Directive (EU) 2019/790 (DSM, Art. 4 text and data mining) - https://eur-lex.europa.eu/eli/dir/2019/790/oj