Learning analytics & dropout early warning

Universities and schools lose students who struggle silently: by the time grades show the problem, support comes too late. Early-warning systems analyse engagement and performance data to estimate who is at risk, explain why, and route a human adviser to reach out early. The technology is mature (classic tabular ML); the hard parts are data integration, fairness, legal basis and getting interventions to actually happen.

Problem & value

• Today: advisers rely on end-of-term grades, teacher intuition or students asking for help. Students who need help most are often the least likely to ask. Tutors cannot monitor hundreds of learners across several systems.
• AI value: continuous risk estimates from many weak signals (late logins, skipped submissions, falling credit pace), prioritised adviser worklists, course-level insight (which module loses people), and evidence for which interventions work.
• Evidence: the research field is well established (e.g. the Open University Learning Analytics Dataset is a public benchmark [1]), but published results show prediction accuracy varies strongly by context and that benefit depends on the intervention, not the score. Treat vendor claims of retention uplift as hypotheses to test locally; no reliable general figure exists.

Variants

• Higher education: dropout and study-delay prediction from enrolment, credit progress, LMS data.
• Schools (secondary/vocational): attendance and grade-based warning for disengagement; stricter rules for minors and parental involvement.
• MOOC / online programmes: course-completion prediction from clickstream.
• Corporate training: completion and engagement monitoring for mandatory courses (see EDU-006); employee-privacy and works-council rules apply.
• Teacher-facing dashboards: descriptive analytics only, no individual scoring (lowest risk).
• Private use: self-tracking of study habits by learners themselves (see EDU-016).

How it works

1. sources (LMS, SIS, grades, attendance)
2. ingestion + identity mapping (pseudonymised student key)
3. feature engineering (engagement trends, credit pace, grade trajectory, time-windowed)
4. model (gradient-boosted trees / logistic regression; per-week or per-term snapshots)
5. calibration + explanation (SHAP-style top factors, fairness metrics per group)
6. adviser dashboard / worklist (human-in-the-loop)
7. outreach (email, meeting offer; optional LLM-drafted message, reviewed by adviser)
8. outcome logging
9. evaluation and retraining

• Prediction: tabular models on time-windowed features remain the workhorse; deep sequence models rarely beat well-built boosted trees on typical institutional data sizes.
• LLM role: summarising a student's situation for the adviser, drafting supportive (never threatening) messages, clustering free-text survey comments. LLMs should not produce the risk score.
• Human-in-the-loop: the score is a prompt for a conversation; advisers decide on contact, and no automatic consequence (grade, admission, funding, exmatriculation) follows from it.

Data preparation

Data comes from several systems: LMS activity logs, the student information system (enrolment, credits, grades), optionally attendance and library or campus data. These typically live in separate databases, are owned by different departments, and use different student identifiers.

Typical gaps: no common key across systems, logs that are kept only briefly, inconsistent grade and status codes, missing outcome labels (who actually dropped out and when), bias from small cohorts, and unclear legal basis and access rights.

Preparation steps: (1) inventory sources and owners (PREP-002, PREP-004); (2) get data out of the systems via exports or APIs (PREP-005); (3) build one tidy table per student and term (PREP-011); (4) harvest labels from historical enrolment records (PREP-020); (5) define the glossary of statuses and KPIs (PREP-012); (6) check purpose and legal basis (PREP-025) and pseudonymise for model development (PREP-022); (7) check fairness across groups. Tools: SQL, Python/pandas, a data warehouse, DataLad or similar versioning.

Rough share of effort: 50-60 percent, usually more than modelling itself. See GUIDE-003.

Implementation paths

Which path for whom

• Public (universities, schools): Path C, because student data is highly sensitive and decisions have real consequences, so a tabular model on own hardware with open-source tooling (Moodle analytics, scikit-learn, Open Learning Analytics) keeps data in-house.
• Enterprise: Path A, because corporate LMS and HR analytics products offer early-warning features for training completion as a commodity function.
• Developer: Path C, because open-source ML stacks and a transparent model with explainable factors are the safest base for a product.

Path A — Buy (recommended for corporate learning)

• LMS-native analytics: Moodle Learning Analytics (open-source, built-in "students at risk of dropping out" model and API for custom models [2]); Canvas/Instructure and Blackboard/Anthology analytics add-ons; D2L Brightspace Performance Plus.
• Student-success platforms: Ellucian, Salesforce Education Cloud, EAB Navigate, Civitas Learning (mostly US-centred; check EU hosting and GDPR terms). EU/DACH options: campus-management vendors (e.g. HISinOne by HIS eG, CAMPUSonline, Ilias/Stud.IP-based projects) and national initiatives; verify current analytics modules per vendor.
• Choose A when: one dominant LMS/SIS, small data-science team, need a supported dashboard.

Path B — Build on cloud APIs

• Data platform: warehouse in an EU region (Azure, AWS Frankfurt, Google Cloud EU, or EU providers such as STACKIT, OVHcloud, IONOS).
• Modelling: AutoML or managed ML services (Azure ML, Vertex AI, SageMaker) on pseudonymised data.
• LLM layer: an EU-region frontier model (Claude, Gemini, GPT via EU endpoints) or Mistral (EU) for message drafting and summaries; send only minimised, pseudonymised context.
• Integration: LTI 1.3 / xAPI / Caliper events from the LMS; IMS-standard data pipelines.

Path C — Self-host (open source; recommended for public institutions)

• Stack: scikit-learn, XGBoost/LightGBM, SHAP for explanations, Fairlearn for group metrics [3], MLflow, dbt/Airflow or Dagster, Superset/Metabase dashboards; Moodle analytics API.
• LLM drafting (optional): an open-weight model such as Mistral Small, Qwen3 or Llama via Ollama or vLLM; HW1 for tabular ML, HW2 for local drafting models.
• Effort: a data engineer plus a data scientist; data stays on campus, which eases consent and works-council negotiations.

Path D — Custom training

Almost always combined with C: train institution-specific models on 3–5 years of cohort history, as generic models transfer poorly across programmes and countries. Needs several thousand student-term records with a clear outcome label (dropout, delay, failed course). Re-validate every term for drift (curriculum changes, post-pandemic effects).

State of the art (2026)

• Models: boosted trees on engineered features with weekly snapshots remain the strong baseline; survival analysis (time-to-dropout) and calibrated probabilities are preferred over raw labels.
• Explainability and fairness are now standard requirements: per-student factor lists and subgroup performance (gender, migration background, first-generation, disability where lawfully processable) must be checked, since models can learn proxies and systematically over- or under-flag groups.
• Intervention science: research increasingly focuses on what happens after the flag (nudges, tutoring, advising) and on A/B or staggered roll-out evaluation rather than on AUC alone.
• LLM assistance shifts effort from reading dashboards to acting: summaries, outreach drafts, conversational access to cohort statistics.
• Limits: self-fulfilling prophecies and labelling effects; low base rates make false positives common; LMS clicks poorly measure real learning; models degrade when courses change; students who study offline look "inactive". Never infer emotions or attention from camera or voice (see AI Act below).

Shared resources & combinations

• Same stack (tabular-ml, time-series): PRED-003 customer churn and PRED-012 employee attrition use the same modelling and monitoring tooling; one MLOps platform can serve all (HW1).
• Combines with: EDU-003 (grading data and feedback), EDU-005 (adaptive paths as the intervention), EDU-004 (AI tutor engagement signals), EDU-008 (integrity data must stay separate from support analytics), EDU-006 (corporate completion tracking).
• Data pipeline: identity mapping, pseudonymisation and consent management are reusable across all student-data use cases.

Effort, cost & KPIs

• Path A: weeks to a term to switch on, mostly licence cost plus change management.
• Path B/C: 4–9 months for data integration, a validated model, dashboard and adviser process; the largest cost is people (data engineering, advisers' time), not compute.
• KPIs: retention/continuation rate versus comparison cohort, share of flagged students contacted within X days, precision/recall at the adviser's capacity, calibration, subgroup error-rate gaps, student-reported helpfulness, adviser workload.

People & change

Involve management and the data owners first (registrar, IT, faculty heads), then advisers and teachers who act on the output, plus student representatives. IT security and IT handle integration; the DPO must be involved from the concept phase: systematic evaluation of people means a DPIA is likely and the legal basis for profiling students needs clear review (GUIDE-005). Under the AI Act, depending on the use, systems that evaluate learning outcomes can fall into the high-risk area (Annex III), with duties from 2 Dec 2027 (GUIDE-014).

Works council: yes where staff are analysed, and teachers' behaviour in the LMS is also logged; the system is objectively suited to monitor behaviour, so §87(1) no. 6 BetrVG (in public institutions the staff council under BPersVG/state law) applies, with §90(1) no. 3 early information (GUIDE-004).

Support AI literacy of advisers (Art. 4 AI Act, GUIDE-006): scores are indications, not verdicts. Typical concerns are stigmatisation and surveillance. Address them with supportive, opt-in outreach, explainable factors, no automatic consequences, and human advisers deciding every contact. Pilot with one faculty and evaluate outcomes before scaling (GUIDE-009).

Risks, regulation & compliance

• EU AI Act: Annex III(3) lists education systems used to determine access or admission, evaluate learning outcomes (including when steering the learning process), assess the appropriate level of education, or monitor prohibited behaviour in tests [4]. A dropout score that only prioritises voluntary adviser support is arguably outside this list, but a system that evaluates or steers a student's learning path, or feeds into decisions on progression, funding or exmatriculation, should be treated as high-risk (risk-management, data governance, logging, human oversight, transparency; high-risk obligations apply from 2 Dec 2027 under the Digital Omnibus on AI, Reg. (EU) 2026/1744). Art. 5 prohibition: emotion recognition in education institutions (e.g. inferring boredom or stress from webcam or voice) is banned except for medical or safety reasons; do not include it.
• GDPR: student data is personal data; for minors extra care. Legal basis typically public task or legitimate interest for universities (national higher-education law), not blanket consent; DPIA required (large-scale profiling), purpose limitation, data minimisation, retention limits. Art. 22 restricts solely automated decisions with significant effect; keep a human decision-maker [5].
• Co-determination and ethics: student representatives, data-protection officer and, for corporate learners, the works council must be involved; publish a plain-language notice and opt-out/appeal route.
• Typical risks and mitigations: stigmatisation (restrict who sees scores; use supportive wording), bias (subgroup audits each term), opaque models (explanations), function creep (separate from disciplinary and integrity use), data breaches (pseudonymisation, access logs).

Getting started

1. Define the decision and intervention first: who contacts whom, with what offer, within what time.
2. Run a DPIA and agree the legal basis, data scope and retention with the data-protection officer.
3. Build a baseline from 3+ years of history; compare a simple logistic model with boosted trees.
4. Pilot in one faculty or programme for one term with a comparison group; audit subgroup fairness.
5. Review the pilot with advisers and students, then decide on scale-up or stop.

Sources

1. Kuzilek, Hlosta, Zdrahal — Open University Learning Analytics Dataset, Scientific Data 2017 — https://www.nature.com/articles/sdata2017171
2. Moodle Docs — Learning analytics (students at risk of dropping out model) — https://docs.moodle.org/en/Learning_analytics
3. Fairlearn project — fairness assessment and mitigation toolkit — https://fairlearn.org
4. Regulation (EU) 2024/1689 (AI Act), Art. 5 and Annex III — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
5. Regulation (EU) 2016/679 (GDPR), Art. 22 and Art. 35 — https://eur-lex.europa.eu/eli/reg/2016/679/oj
6. SHAP documentation — https://shap.readthedocs.io
7. scikit-learn documentation — https://scikit-learn.org