Smart sensors & TinyML on microcontrollers

Sending every raw sensor sample to the cloud costs battery, bandwidth and privacy. TinyML runs small, quantised models (kilobytes to a few megabytes) on microcontrollers (MCUs) or MCU-class NPUs right at the sensor, so a node only transmits "bearing anomaly" or "wake word heard" instead of a raw stream. Typical benefits: years of battery life, millisecond latency, offline operation and data that never leaves the device.

Problem & value

• Today: sensor nodes either use fixed thresholds (many false alarms, missed subtle faults) or stream raw data to gateways/cloud (costly radio, power, storage, latency, privacy exposure).
• AI value: learned detectors on vibration, sound, current or motion find patterns thresholds cannot, and local inference cuts transmitted data to events. Audio and camera data can be reduced to labels on the device.
• Why now: MCUs with DSP extensions and tiny NPUs (Arm Cortex-M55/M85 with Ethos-U55/U65, STM32N6 with Neural-ART, Renesas RA8, Infineon PSOC Edge, Nordic nRF54/Axon-class parts) make inference cheap, and mature toolchains hide most low-level work [1][2][3][4].
• No universal benefit figure exists; always benchmark battery life, accuracy and bill-of-materials on your own device.

Variants

• Predictive maintenance nodes: vibration/current anomaly detection on motors, pumps, fans (see PRED-006, ANOM-005).
• Acoustic & keyword spotting: machine sound monitoring, glass-break, wake words, cough/snore classification.
• Wearables & motion: gesture, activity, fall detection; exoskeleton/gait control (ROBO-018).
• Agriculture & environment: livestock collars, soil/pest sensing, wildlife audio (PRED-044).
• Smart home & buildings: presence, occupancy, appliance signatures (ROBO-012).
• Structures & infrastructure: battery-powered strain/tilt nodes (PRED-046).
• Private makers: Arduino/ESP32 gesture or sound projects.

How it works

1. sensor (IMU / mic / ADC / camera)
2. DSP preprocessing on MCU (filtering, FFT, MFCC, windowing)
3. quantised NN (int8) or classical model (tree, k-means, anomaly)
4. decision / event (threshold, debouncing, state machine)
5. local action (actuator, wake main CPU) and/or compact uplink (LoRaWAN, BLE, Thread, NB-IoT, MQTT)
6. fleet backend: monitoring, retraining, OTA model update

• Model types: small 1D-CNNs, depthwise-separable CNNs, small RNN/transformer variants, autoencoders for "normal-only" anomaly detection, and classical ML (random forest, isolation forest) which often wins on tabular sensor features.
• Compression: post-training or quantisation-aware int8 quantisation, pruning, knowledge distillation, operator fusion; a model must fit RAM/flash alongside firmware.
• Human-in-the-loop: domain experts label events and judge anomaly alerts; thresholds and retraining decisions are reviewed by engineers; safety actions never rely on an unvalidated model alone.

Data preparation

The key data are labelled sensor time series (vibration, current, temperature, audio, images) from the real device or machine, plus context such as operating state and fault records. In an SMB these typically sit in PLC/SCADA historians, maintenance logs, CSV exports, Excel sheets and in the heads of service technicians.

Typical gaps: no labelled fault examples (faults are rare), inconsistent sampling rates and clocks, sensor drift, data only from good conditions, labels missing or only in free-text maintenance notes, and data scattered across machines.

Preparation steps: (1) define what to detect and the label categories (PREP-019); (2) run a short structured recording campaign with normal and, where possible, deliberately induced fault states; (3) harvest labels from maintenance and work-order records (PREP-020); (4) clean, resample and align time series into tidy tables (PREP-011); (5) hold out an evaluation set with experts (PREP-018). Tools: Edge Impulse data forwarder, Python/pandas, simple MQTT or serial loggers.

Rough share of effort: 30-40 percent, mostly the recording campaign and labelling. See also PREP-005 for data extraction and GUIDE-003 for the readiness check.

Implementation paths

Which path for whom

• SMB: Path B — managed TinyML platforms (e.g. Edge Impulse) let a small team train and deploy models to standard microcontrollers without ML specialists.
• Enterprise: Path C — a shared open-source toolchain (TensorFlow Lite Micro, own MLOps) across many products avoids per-device licence costs and keeps sensor data in-house.
• Developer: Path C — open-source frameworks and open firmware are the natural stack for embedded ML.

Path A — Buy (off-the-shelf)

• Pre-built smart sensors: condition-monitoring sensors with on-board ML from industrial vendors, e.g. Bosch (EU) XDK/sensor modules, STMicroelectronics (EU) MEMS sensors with Machine Learning Core / Intelligent Sensor Processing Unit, Infineon (EU) XENSIV sensors, Siemens (EU) and SKF (EU) condition monitoring, plus Syntiant, Aizip or Nordic Edge AI-class parts.
• Choose A when: the sensing task is standard (vibration on motors, occupancy, basic audio events) and you do not want to own firmware or ML.

Path B — Build on cloud APIs / platforms

• MLOps platforms for embedded ML: Edge Impulse (now part of Qualcomm), Nota/Neuton/Imagimob Studio (Imagimob is Infineon, EU), STMicroelectronics ST Edge AI Developer Cloud and STM32Cube.AI / ST Edge AI Core (EU), Nordic Edge AI Lab, Renesas Reality AI. They cover data capture, DSP block design, training, quantisation, benchmarking on target boards and code export [1][2][5].
• Cloud role: collect labelled field data, train, store fleet telemetry (AWS IoT Core/Greengrass, Azure IoT Hub, EU regions; EU-hosted options like Scaleway, OVHcloud, Open Telekom Cloud), and distribute OTA updates (Memfault, Golioth, Mender).
• Choose B when: you have embedded engineers but want fast iteration; the default for most SMB product teams.

Path C — Self-host (open source)

• Runtimes: LiteRT for Microcontrollers (formerly TensorFlow Lite Micro), ExecuTorch (PyTorch, supports Arm Ethos-U and Cortex-M backends), microTVM/Apache TVM, Arm CMSIS-NN, emlearn (classical ML in C), ONNX-based exports, Zephyr RTOS and FreeRTOS as OS layers [3][4][6].
• Training: PyTorch/Keras/scikit-learn, quantisation with Model Optimizer/AIMET-type tools, MLPerf Tiny as benchmark suite [7].
• Hardware: development on a laptop/HW1 class; target boards such as STM32, nRF, ESP32-S3, RP2350, Alif Ensemble (Cortex-M55 + Ethos-U55), Raspberry Pi Pico 2.
• Effort: firmware plus ML skills; mainly embedded engineering time.

Path D — Custom training

Most real projects end here: collect your own device-specific data (sensor placement, noise, machine variants), design features, train and quantise per device family. Needs hundreds to thousands of labelled windows for classification; for "normal-only" anomaly models, representative recordings over operating states, seasons and wear. Neural architecture search (e.g. MCUNet-style) and distillation are worthwhile for tight RAM budgets [8].

State of the art (2026)

• MCU NPUs are mainstream: int8 inference for vision at low resolution, keyword spotting and anomaly detection runs in milliwatts; Ethos-U, Neural-ART and similar accelerators give large speedups over plain Cortex-M cores. Transformer-style and small-VLM workloads remain mostly beyond MCUs; they target edge-Linux/NPU boards (HW1 class such as Jetson Orin Nano, Hailo).
• Toolchains converge: ExecuTorch and LiteRT replace hand-written C inference; vendor tools profile latency, RAM and energy per layer on real boards; MLPerf Tiny standardises comparison [6][7].
• Trend: data-centric workflows (synthetic augmentation, auto-labelling with larger cloud models, then distilling to the MCU), on-device continual calibration, and tiny foundation-style time-series encoders for few-shot adaptation.
• Limits and failure modes: distribution shift (new machine, new mounting, noise) silently degrades accuracy; int8 quantisation can cut accuracy; unsupervised anomaly detectors raise false alarms without ground truth; tiny RAM limits context length; OTA updates and secure boot add effort; energy claims from datasheets differ from field battery life.

Shared resources & combinations

• Shared capabilities: edge-runtime, time-series and anomaly-detection pipelines are reused across PRED-006 (predictive maintenance), ANOM-005 (process anomalies), PRED-014 (soft sensors), PRED-044 (livestock), PRED-046 (structural monitoring).
• Combinations: TinyML as the first-stage trigger that wakes a bigger edge model or gateway (ROBO-016 safety sensing on machines, ROBO-012 smart home hubs); wearables for ROBO-018.
• Shared infrastructure: MQTT/LoRaWAN gateways, OTA and fleet-management service, a labelled-data store, and a cloud training environment serve many device types.

Effort, cost & KPIs

• Path A: days to weeks to deploy; per-sensor hardware plus optional subscription.
• Path B: typically 2-6 months for a first product-grade node (data collection, model, firmware, OTA); platform licences or free tiers; cost is mostly engineering and field data collection.
• Path C/D: longer, with embedded and ML specialists; per-unit BOM savings pay off at volume.
• KPIs: precision/recall per event, false alarms per device-week, inference latency, RAM/flash footprint, energy per inference and battery life, share of data not transmitted, OTA success rate, fleet-wide drift indicators.

People & change

Involve management for the business case, the department lead (maintenance or product management) as owner, IT and IT security for device update and connectivity, and data owners of PLC and maintenance data. Service technicians are the experts who validate labels, so involve them from the start.

Works council: depends. Sensors on machines are mostly uncritical, but data tied to a machine operator, shift or individual performance can be suited to monitor behaviour (§87(1) no. 6 BetrVG, objective suitability suffices). Inform the works council in good time (§90(1) no. 3) and agree that data is used for machine condition only. In pure consumer or product embedded use, no works council is involved.

A DPIA is normally not needed for machine data without personal reference; consult the DPO if audio or camera sensors can capture people. Support AI literacy (Art. 4 AI Act) with short training for technicians on what alerts mean and their false-alarm rates. Pilot on one or two machines with a motivated maintenance team, communicate that the sensor supports rather than replaces their judgement, and address fears about monitoring. Alerts recommend, humans decide on maintenance actions.

Guides: GUIDE-009, GUIDE-008, GUIDE-004.

Risks, regulation & compliance

• EU AI Act: usually minimal risk (AI literacy, Art. 4). It becomes high-risk when the sensor AI is a safety component of a product under Annex I legislation (machinery, medical devices; Annex I obligations apply from 2 Aug 2028 per Digital Omnibus Reg. (EU) 2026/1744, radio equipment with safety function, toys) or in Annex III areas; biometric identification, emotion inference at work/school and similar uses are restricted or prohibited under Art. 5, so audio/vision nodes must not drift into those purposes [9].
• GDPR: microphones, cameras, wearables and presence sensors can process personal data (employees, residents, health data). Local inference with only labels leaving the device supports data minimisation and privacy by design, but the device may still process personal data; document purpose, DPIA where needed, works-council involvement for workplaces.
• Product law: Cyber Resilience Act (secure updates, vulnerability handling), Radio Equipment Directive cybersecurity requirements, new Machinery Regulation 2023/1230 for safety functions, MDR for medical wearables [10].
• Technical risks: model theft/tampering on device (use secure boot, encrypted storage), adversarial sound/signals, silent drift. Mitigate with monitoring, canary rollouts, rollback and fallbacks to conventional thresholds.

Getting started

1. Pick one measurable event (e.g. bearing wear, wake word) and define target recall, false-alarm rate and battery life.
2. Buy a dev kit with the target MCU and sensor; record 1-2 weeks of real data with ground truth.
3. Prototype in Edge Impulse or ST Edge AI; compare a classical model with a small CNN.
4. Quantise to int8, benchmark RAM/latency/energy on the board, then integrate into firmware with a threshold fallback.
5. Pilot on 10-50 devices with OTA and drift logging before scaling.

Sources

1. Edge Impulse documentation — https://docs.edgeimpulse.com/
2. STMicroelectronics — ST Edge AI Suite — https://stm32ai.st.com/
3. LiteRT for Microcontrollers (Google AI Edge) — https://ai.google.dev/edge/litert/microcontrollers/overview
4. Arm — Ethos-U NPU and CMSIS-NN — https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u55
5. Nordic Semiconductor — Edge AI (nRF54LM20B with Axon NPU, Nordic Edge AI Lab; 6 Jan 2026) — https://www.nordicsemi.com/Nordic-news/2026/01/nRF54L-Series-SoC-with-NPU-and-Nordic-Edge-AI-Lab-make-on-device-intelligence-easily-accessible
6. ExecuTorch (PyTorch) — https://pytorch.org/executorch/
7. MLCommons — MLPerf Tiny benchmark — https://mlcommons.org/benchmarks/inference-tiny/
8. Lin et al. — MCUNet: Tiny Deep Learning on IoT Devices (NeurIPS 2020) — https://arxiv.org/abs/2007.10319
9. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
10. Regulation (EU) 2024/2847 (Cyber Resilience Act) — https://eur-lex.europa.eu/eli/reg/2024/2847/oj