Hands-free voice control for industrial work (voice picking, maintenance)

Order pickers, goods-in staff and maintenance technicians need both hands and their eyes for the job, not for a handheld scanner or a paper checklist. Voice systems read the next task into a headset and take spoken confirmations ("check digit 42", "done", "defect: bearing noise"). Classic voice picking is a mature, proven technology; modern ASR and LLMs now add free-form notes, multilingual workforces and guided maintenance dialogue. The main new constraint in the EU is worker monitoring law.

Problem & value

• Today: pick-by-paper or handheld RF scanners force workers to stop, look at a screen and press keys; maintenance technicians in gloves or on ladders write notes afterwards from memory. Multilingual crews and temporary staff struggle with screen-based UIs.
• AI value: hands-free, eyes-free work; immediate and structured data capture (fewer transcription errors, real-time stock accuracy); faster onboarding through spoken guidance in the worker's language; free-form fault descriptions that become structured work-order text.
• Evidence: vendors and logistics trade bodies report productivity and accuracy gains versus paper picking, but figures depend on process and baseline; measure your own pilot rather than relying on vendor claims [1][2].

Variants

• Voice picking / put-away / replenishment: dialogue-driven, closed vocabulary (digits, locations, a few commands).
• Goods-in, loading, inventory counting: voice counting and quantity confirmation.
• Maintenance and inspection rounds: checklist walk-through by voice, spoken readings, free-text fault notes, optional manual lookup (see KNOW-004).
• Production and quality: hands-free reporting of scrap, setup steps or defects; clean-room and food/pharma environments where touching devices is undesirable.
• Multimodal wearables: voice combined with smart glasses or head-mounted displays (see CONV-027).
• Private use: not relevant; adjacent consumer uses are in-car or smart-home assistants (CONV-014).

How it works

1. headset microphone (noise-cancelling, often Bluetooth)
2. on-device audio front-end (noise suppression, VAD)
3. ASR: closed-grammar / keyword recogniser or compact neural ASR (edge or gateway)
4. dialogue manager (state machine bound to the task: location
5. check digit
6. quantity)
7. WMS / ERP / CMMS integration (task fetch, confirmation, exception)
8. TTS prompts back to headset (multilingual)
9. optional LLM layer: free-text fault notes
10. structured work-order fields; manual Q&A

• Deterministic core: voice picking works because the dialogue is a state machine with a small vocabulary; recognition errors are caught by check digits and confirmation prompts. This core should stay deterministic even when LLMs are added.
• Worker-specific adaptation: classic engines train a short voice profile per worker; newer neural ASR reduces or removes this enrolment step. Do not confuse this with voice biometrics (SPEECH-010).
• LLM layer (optional): converts free speech such as "pump three leaking at the seal, needs new gasket" into structured fields and retrieves manual passages. A human confirms before the work order is closed.
• Human-in-the-loop: the worker confirms every action; supervisors handle exceptions (short picks, damaged goods).

Data preparation

Data needed: master data for the voice dialogue, mainly item, location and check-digit data from the WMS or ERP, plus a vocabulary of product names, bin codes and abbreviations. Recorded sample speech from the real environment (noise, headsets, accents) is needed for testing. Master data typically lives in the ERP or WMS; terminology is often only in employees' heads or on paper pick lists.

Typical gaps: inconsistent item descriptions and location naming, missing check digits, scattered abbreviations, no recordings from the real hall noise, and unclear access rights from the voice device to the WMS.

Preparation steps: 1. Clean location and item master data and unify naming (PREP-005 for export from ERP, PREP-011 if data sits in spreadsheets). 2. Build a term and abbreviation list including dialect variants (PREP-015). 3. Record test utterances from several workers in the real noise (PREP-018). 4. Collect corrections when the system mishears (PREP-021). 5. Agree what is stored: command logs without audio retention (PREP-024).

Effort share: low, about 15 to 25 percent, mostly master data clean-up and vocabulary tuning. The effort rises if paper-based processes must be digitised first (PREP-008).

Implementation paths

Which path for whom

• smb: Path A, because dedicated voice-picking suites integrate with common WMS and ERP systems and the process is standard.
• enterprise: Path B, because multi-site operations with own WMS, several languages and dialects need custom integration and cost control on seats.
• developer: Path C, because Whisper-class ASR and small LLMs run on HW1 edge devices and allow offline use in noisy halls.

Path A — Buy (recommended for most SMBs)

• Dedicated voice-picking suites: Honeywell Voice (Vocollect), Zebra (voice via Workcloud/Wavelink ecosystem), Körber Supply Chain Software, Dematic and Blue Yonder-class WMS with voice modules, plus WMS-native voice modules from SAP EWM partners and other WMS vendors. EU-based providers include Körber, SSI Schäfer's software division and numerous German WMS vendors with voice add-ons; verify current offerings with each vendor.
• Maintenance/inspection apps: CMMS/EAM vendors (IBM Maximo, SAP, Fiix, and EU vendors such as Infraspeak or Limble-class tools) with speech input, and wearable-computing vendors (RealWear, Vuzix) offering voice-driven head-mounted tablets for hands-free work.
• Choose A when: standard picking or inspection flows, an existing WMS/CMMS with a supported voice connector, and a need for rugged certified hardware.

Path B — Build on cloud APIs

• Streaming ASR: Azure AI Speech (EU regions, custom speech and phrase lists), Google Cloud Speech-to-Text, AWS Transcribe (Frankfurt), Deepgram, Speechmatics (UK/EU), Mistral Voxtral (EU provider), OpenAI realtime/transcription models. Phrase lists and custom vocabularies improve article and location recognition.
• TTS: Azure Neural voices, Google, ElevenLabs, or EU providers.
• Caveat: industrial noise, unstable Wi-Fi in racking aisles and latency make pure cloud ASR risky for the core dialogue; use cloud for free-text notes and edge for command recognition.
• Choose B when: you need free-form notes, many languages or custom integration.

Path C — Self-host (open source)

• Open ASR: Whisper (large-v3 / turbo) via faster-whisper or whisper.cpp, NVIDIA Parakeet/Canary (NeMo), Vosk and sherpa-onnx for small, offline, embedded recognisers with custom grammars; Kyutai and Voxtral open-weight models for streaming.
• Keyword/command recognition: small CNN-based keyword-spotting models or grammar-constrained decoding on Raspberry Pi- or Android-class devices.
• TTS: Piper, Kokoro, Coqui-derived models, offline on-device.
• Hardware: HW1 is enough for command grammars and compact ASR on an edge gateway or Android device; HW2 if one server should serve free-text transcription for many sites.

Path D — Custom training

Worth it for dialects, rare languages, heavy machine noise or domain terms (part numbers, chemical names). Fine-tune an open ASR model on a few tens of hours of in-domain audio recorded with the real headset and noise; correction logs from production are the best data source. Handle recordings as personal data.

State of the art (2026)

• Closed-grammar voice dialogue is a solved, rugged technology; the cutting edge is neural ASR that tolerates noise and accents without per-worker training, plus LLMs for free-text and guidance.
• Open models such as Whisper-class, Parakeet and Voxtral give good general accuracy, but robustness to forklift noise, hearing protection, masks and non-native speakers must be tested on site.
• Failure modes: short, similar-sounding digits and codes; background speech from colleagues; hallucinated text on silence or noise in Whisper-type models; latency over poor networks; wake-word false triggers. Mitigate with check digits, confirmation prompts, VAD, push-to-talk and grammar constraints.
• Safety: voice must never be the only channel for safety-critical commands (e.g. machine stop); keep hardware emergency stops and treat hearing-protection and situational awareness as design constraints.

Shared resources & combinations

• Same stack (asr + tts + edge-runtime): SPEECH-001 and SPEECH-005 (dictation), SPEECH-006 (TTS), SPEECH-009 (audio event detection from the same microphones), CONV-014 (embedded voice).
• Upstream/downstream: OPT-006 (pick-path and slotting optimisation feeds the pick list), ROBO-002 (AMRs combined with picker voice dialogue), KNOW-004 (manual assistant for technicians), PRED-006 (voice-reported faults enrich predictive maintenance), CONV-027 (visual remote assistance).
• Caution with SPEECH-010: voice login or speaker identification is biometric processing with extra legal requirements.

Effort, cost & KPIs

• Path A: weeks for a pilot area; costs are headsets/devices, licences per user or per device, and WMS integration.
• Path B: 2-4 months for a robust edge/cloud hybrid including WMS integration and dialogue design.
• Path C: 3-6 months; mainly dialogue design, audio testing in real environments and device management.
• KPIs: picks per hour, pick error rate, training time for new staff, recognition error and repeat rate per command, ratio of voice dialogue time to travel time, free-text-to-structured accuracy, worker satisfaction.

People & change

Involve management and the logistics department lead first to define the goal (accuracy, speed, ergonomics). End users are decisive: pickers should test headsets and voice commands early and co-define the command vocabulary. IT integrates the system with the WMS and manages devices; the DPO checks what is logged and whether voice recordings are kept.

Works council: yes. Voice systems log each command with timestamps and worker ID, so they are suited to monitor performance (§87(1) no. 6 BetrVG; BAG: objective suitability is enough). The works council must also be informed under §90(1) no. 3, and may call an expert under §80(3). Typical works agreement: no individual performance ranking from voice data, defined retention, no audio storage. Public sector: staff council. A DPIA is usually not required if no audio is kept, but check with the DPO.

Communicate that the aim is relief (hands free, fewer errors), not control. Pilot with one shift or zone and AI champions from the workers. Typical concerns are surveillance, hygiene of shared headsets, accents and noise; respond with personal headsets, accent tuning, and a manual fallback. Workers can always override the system. AI literacy measures should support staff in using it confidently (Art. 4 AI Act).

See GUIDE-004, GUIDE-008 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: a pure command-and-control system is typically minimal risk; AI literacy (Art. 4) applies. Risk rises if voice data is used to monitor, evaluate or allocate tasks to workers (Annex III, employment, point 4: high-risk). Art. 5(1)(f) prohibits emotion recognition in the workplace (except medical or safety reasons): do not add stress, fatigue or mood detection from worker voices for performance management.
• GDPR and labour law: audio and productivity data are employee personal data. Define purpose limitation (task execution, not performance surveillance), minimise retention (process audio on device, store transcripts only if needed), run a DPIA for continuous monitoring, and involve the works council. In Germany, co-determination under §87(1) no. 6 BetrVG applies to technical systems capable of monitoring staff. Voice profiles used to identify workers are biometric data (Art. 9 GDPR) if used for unique identification.
• Safety: workplace noise, hearing protection and ATEX zones for devices (explosive atmospheres) need certified hardware.
• Mitigations: on-device processing, aggregated KPIs rather than individual rankings, works agreement, confirmation prompts, hard-wired safety functions.

Getting started

1. Pick one process (e.g. a picking zone or an inspection round) and record baseline productivity and error rate.
2. Record real audio in the actual environment with candidate headsets; test vocabulary and noise.
3. Check your WMS/CMMS for supported voice connectors before considering a custom build.
4. Agree purpose, retention and KPI use with the works council and complete a DPIA before the pilot.
5. Run a 4-8 week pilot with a control group; add LLM-based free-text notes only after the core dialogue is stable.

Sources

1. Honeywell — Voice solutions for warehouse and logistics — https://sps.honeywell.com/us/en/products/productivity/voice-solutions
2. Zebra Technologies — Warehouse workflows and voice — https://www.zebra.com/
3. RealWear — hands-free wearable computing — https://www.realwear.com/
4. OpenAI Whisper — https://github.com/openai/whisper
5. sherpa-onnx — https://github.com/k2-fsa/sherpa-onnx
6. Vosk speech recognition toolkit — https://alphacephei.com/vosk/
7. Mistral AI — Voxtral — https://mistral.ai/news/voxtral
8. Azure AI Speech — custom speech and phrase lists — https://learn.microsoft.com/azure/ai-services/speech-service/
9. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
10. Gesetze im Internet — BetrVG §87 — https://www.gesetze-im-internet.de/betrvg/__87.html