Catalog / Speech & Audio / SPEECH-017

AI hearing aids & hearables: on-device speech-in-noise enhancement

Deep neural networks on milliwatt-class chips in hearing aids and hearing-aid-capable earbuds separate speech from noise in real time for people with hearing loss.

Parts needed to run it yourselfSpeechPrediction & time seriesVision

Understanding speech in noisy places such as restaurants or family gatherings is the main unmet need of people with hearing loss. Classic hearing aids amplify and use directional microphones; since 2024 premium devices run a deep neural network (DNN) on a dedicated low-power chip that separates speech from background noise in real time, with latency of a few milliseconds and no cloud round trip. Earbuds with a hearing-aid mode extend this to a much larger, mostly older user group.

Problem & value

  • Today: hearing loss is common in older adults and typically reduces speech understanding most in noise. Hearing aids provide frequency-dependent gain, compression, feedback cancellation and beamforming. Beamforming and classic noise reduction help little when speech and noise come from similar directions or the noise is itself speech.
  • AI value: DNNs trained on large corpora of noisy speech estimate and keep the speech component, suppress the rest, and classify the scene to choose settings automatically. Manufacturers report clear gains in speech-in-noise tests (Sonova claims users more than doubled speech understanding in noise in a clinical study and cites about 10 dB SNR improvement for its Sphere mode [1][2]). These are vendor figures; independent evidence is still thinner and should be checked per product.
  • Access shift: in the US, FDA created an over-the-counter (OTC) hearing-aid category for mild-to-moderate loss (effective Oct 2022), and in 2024 FDA authorised a hearing-aid feature for AirPods Pro 2 [5][6]. In the EU, hearing aids remain regulated medical devices sold mostly via audiologists/acousticians (with reimbursement schemes that differ by country), while consumer earbuds offer hearing-support features of varying regulatory status.

Variants

  • Premium prescription hearing aids with onboard DNN: e.g. Phonak Audéo Sphere Infinio with the dedicated DEEPSONIC chip next to the ERA chip (Sonova, CH/EU) [1][2][3]; Oticon Intent (Demant, DK/EU) with an onboard DNN plus sensor-based "user intent" context [4].
  • OTC/self-fit hearing aids and hearing-aid-capable earbuds (AirPods Pro 2 hearing-aid feature, FDA-authorised 2024; Sennheiser/Sonova consumer lines; startup OTC devices) [5][6].

How it works

  1. 2-4 microphones per ear (+ binaural link between ears)
  2. low-latency filterbank / short-frame encoder (≈ few ms algorithmic delay)
  3. DNN on NPU: speech/noise mask or direct waveform estimation
  4. classical stage: wide dynamic range compression per audiogram, feedback cancellation
  5. output stage / receiver side paths: scene classifier (speech, music, wind, car)
  6. program/strength selection sensors (IMU, head movement)
  7. context; companion app
  8. fitting, remote tuning
  • Core model: small convolutional/recurrent or hybrid networks quantised to int8 or lower, running in the milliwatt range so a battery lasts a full day (battery life varies by model; check manufacturer specifications [3]). The whole chain must stay in the low-millisecond range, otherwise comb-filter and echo effects occur between direct and processed sound.
  • Training (offline, cloud): large sets of clean speech mixed with realistic noise, simulated rooms and hearing-loss-specific objectives. Deployed models are updated with firmware; manufacturers state they use data from real-world fittings to refine them [3].
  • Human-in-the-loop: the hearing-care professional performs audiometry, fitting and verification; the user adjusts strength and programs. AI never changes the prescription without the professional's or user's consent path defined in the fitting software.

Data preparation

For buyers there is hardly any data work: the audiogram and fitting parameters live in the fitting software of the hearing-care professional, usage logs in the companion app. Gaps are mostly consent, not quality: who may see fitting and usage data, and is it shared with the manufacturer for model improvement.

For manufacturers and developers data preparation is the dominant effort (roughly 50-60 % of the project): - Speech and noise corpora: clean speech plus realistic noise, rooms and hearing-loss simulations; check licences and consent of all recordings. - Labels and test sets: listening tests and speech-in-noise measurements (HINT/OLSA/Matrix) with domain experts, see PREP-018. - Real-world fitting data: pseudonymise before reuse (PREP-022) and check purpose and legal basis (PREP-025), because hearing data is health data. - Tools: audio augmentation scripts, DNSMOS/PESQ-type metrics, versioned datasets.

Clinics should additionally inventory where patient audiograms are stored (practice software, paper, USB) with PREP-001 and PREP-003.

Data preparation

For buyers there is hardly any data work: the audiogram and fitting parameters live in the fitting software of the hearing-care professional, usage logs in the companion app. Gaps are mostly consent, not quality: who may see fitting and usage data, and is it shared with the manufacturer for model improvement.

For manufacturers and developers data preparation is the dominant effort (roughly 50-60 % of the project): - Speech and noise corpora: clean speech plus realistic noise, rooms and hearing-loss simulations; check licences and consent of all recordings. - Labels and test sets: listening tests and speech-in-noise measurements (HINT/OLSA/Matrix) with domain experts, see PREP-018. - Real-world fitting data: pseudonymise before reuse (PREP-022) and check purpose and legal basis (PREP-025), because hearing data is health data. - Tools: audio augmentation scripts, DNSMOS/PESQ-type metrics, versioned datasets.

Clinics should additionally inventory where patient audiograms are stored (practice software, paper, USB) with PREP-001 and PREP-003.

Data preparation

For buyers there is hardly any data work: the audiogram and fitting parameters live in the fitting software of the hearing-care professional, usage logs in the companion app. Gaps are mostly consent, not quality: who may see fitting and usage data, and is it shared with the manufacturer for model improvement.

For manufacturers and developers data preparation is the dominant effort (roughly 50-60 % of the project): - Speech and noise corpora: clean speech plus realistic noise, rooms and hearing-loss simulations; check licences and consent of all recordings. - Labels and test sets: listening tests and speech-in-noise measurements (HINT/OLSA/Matrix) with domain experts, see PREP-018. - Real-world fitting data: pseudonymise before reuse (PREP-022) and check purpose and legal basis (PREP-025), because hearing data is health data. - Tools: audio augmentation scripts, DNSMOS/PESQ-type metrics, versioned datasets.

Clinics should additionally inventory where patient audiograms are stored (practice software, paper, USB) with PREP-001 and PREP-003.

Implementation paths

Which path for whom

  • Private: Path A. Certified, reimbursable hearing aids or hearing-capable earbuds with professional fitting are the only safe and supported way to get the benefit.
  • Enterprise: Path A. Clinics, hearing-care chains and employers procure certified devices; own development only pays off for manufacturers (Path D, MDR programme).
  • Developer: Path C. Open speech-enhancement models (e.g. RNNoise, DeepFilterNet) run on HW1 and are enough for prototypes and research.

Which path for whom

  • Private: Path A. Certified, reimbursable hearing aids or hearing-capable earbuds with professional fitting are the only safe and supported way to get the benefit.
  • Enterprise: Path A. Clinics, hearing-care chains and employers procure certified devices; own development only pays off for manufacturers (Path D, MDR programme).
  • Developer: Path C. Open speech-enhancement models (e.g. RNNoise, DeepFilterNet) run on HW1 and are enough for prototypes and research.

Which path for whom

  • Private: Path A. Certified, reimbursable hearing aids or hearing-capable earbuds with professional fitting are the only safe and supported way to get the benefit.
  • Enterprise: Path A. Clinics, hearing-care chains and employers procure certified devices; own development only pays off for manufacturers (Path D, MDR programme).
  • Developer: Path C. Open speech-enhancement models (e.g. RNNoise, DeepFilterNet) run on HW1 and are enough for prototypes and research.

Path A — Buy (default for end users and clinics)

  • Hearing-care route: prescription devices from Sonova/Phonak (CH), Demant/Oticon and Philips-brand devices (DK), WS Audiology/Signia and Widex (DK/DE/SG), GN/ReSound (DK), Starkey (US). Several are EU-headquartered; choose by speech-in-noise evidence, fitting software, service and reimbursement fit.
  • Consumer route: AirPods Pro 2 hearing-aid feature (feature availability varies by country and regulatory status) and OTC devices for mild-to-moderate loss. Not a substitute for an audiological examination: hearing loss can be a symptom of treatable conditions.
  • Choose A: always, unless you are a device manufacturer.

Path B — Build on cloud APIs

  • Cloud processing is generally unsuitable for the live signal path (latency, connectivity, power, privacy). Cloud is used for: model training, fitting assistants, remote fine-tuning sessions, fleet analytics and companion-app services.
  • Phone-side processing has higher latency (often tens of ms): fine for captioning (SPEECH-004), not for open-ear hearing aids.

Path C — Self-host (open source)

  • Research and prototyping: open speech-enhancement models and toolkits such as DeepFilterNet, RNNoise, Demucs-based denoisers, and ESPnet/SpeechBrain recipes; the Clarity Challenges (UK) provide hearing-aid-specific enhancement tasks, data and baselines. Evaluation metrics: PESQ, STOI/ESTOI, HASPI/HASQI (hearing-loss-aware).
  • Deployment targets: microcontrollers/DSPs with NPU (Arm Cortex-M with Ethos-U, vendor DSPs), via ONNX/TFLite Micro/ExecuTorch; HW1 is sufficient for inference, a single GPU (HW2) for training small models.

Path D — Custom training

Relevant for manufacturers and research groups: train speech-enhancement networks on proprietary noise/speech corpora, quantisation-aware training, hearing-loss-aware loss functions, personalisation (e.g. few-shot adaptation to the wearer's voice or frequent partners). Data needs are high (hundreds to thousands of hours plus listening tests with hearing-impaired participants). Required: audiological test lab, clinical validation, MDR quality system.

State of the art (2026)

  • First generation of onboard DNN hearing aids shipped in 2024 (Phonak Sphere Infinio with dedicated DNN chip; Oticon Intent and Oticon Zeal with onboard DNN and sensors) and has been refined through firmware and newer product generations [1][3][4]. Competitors have followed with their own DNN platforms.
  • Trends: dedicated NPUs inside hearing-aid SoCs, Bluetooth LE Audio/Auracast for streaming, binaural processing, own-voice processing, scene-aware strength control, and research on target-speaker extraction with eye or head cues.
  • Limits and failure modes: artefacts when speech and noise are similar (competing talkers); latency/power trade-offs force small models; suppression of wanted sounds (music, alarms, traffic); benefit depends on degree of loss and cognitive factors; independent head-to-head trials are scarce and vendor claims come from controlled tests. Fit quality still dominates real-world outcome.

Shared resources & combinations

  • Shared capabilities: audio-classification and edge-runtime with SPEECH-009 (acoustic event detection) and generic TinyML; speech-enhancement models with SPEECH-008 (offline/cloud audio cleanup).
  • Combinations: SPEECH-004 live captions as a complement in very noisy settings; SPEECH-018 for people who also have speech impairments; PERS-007 as an assistive-technology sibling; CONV-016 for companion devices for older people. Fall detection or health sensing in hearables is a separate medical/consumer topic.

Effort, cost & KPIs

  • End users (Path A): prescription devices are priced per pair in the range of several thousand EUR, partly reimbursed by statutory insurers depending on country; OTC devices and earbuds cost far less (check current prices).
  • Manufacturers (Path D): multi-year programme: chip/NPU co-design, model training, clinical studies, MDR conformity assessment with a notified body, service network.
  • KPIs: speech reception threshold in noise (SRT, e.g. HINT/OLSA/Matrix tests), SNR improvement, speech intelligibility index, end-to-end latency, battery life, listening effort, user satisfaction (e.g. questionnaires such as APHAB/SSQ), return rate and daily use hours.

People & change

Involve management and the department lead (clinic or product owner) first, suppliers (device makers) and legal-compliance (MDR, liability) early, and the DPO before any fitting or usage data leaves the practice or the app: health data needs a clear legal basis and an AVV, and a DPIA is likely when data is reused for model training (GUIDE-005).

Works council: usually no. The devices serve customers or patients, not employee monitoring. It becomes relevant only if an employer rolls out hearables to staff and logs usage (§87(1) no. 6 BetrVG), then see GUIDE-004.

AI literacy: support staff and fitters with short training on what the noise reduction can and cannot do (Art. 4 AI Act, GUIDE-006). Communicate openly with users: the professional stays in control of the prescription, users can adjust or switch off programs. Start with a small pilot group of experienced users and collect questionnaire feedback (GUIDE-008). Typical concern: "the device hears everything"; answer with on-device processing and transparent data sharing choices.

People & change

Involve management and the department lead (clinic or product owner) first, suppliers (device makers) and legal-compliance (MDR, liability) early, and the DPO before any fitting or usage data leaves the practice or the app: health data needs a clear legal basis and an AVV, and a DPIA is likely when data is reused for model training (GUIDE-005).

Works council: usually no. The devices serve customers or patients, not employee monitoring. It becomes relevant only if an employer rolls out hearables to staff and logs usage (§87(1) no. 6 BetrVG), then see GUIDE-004.

AI literacy: support staff and fitters with short training on what the noise reduction can and cannot do (Art. 4 AI Act, GUIDE-006). Communicate openly with users: the professional stays in control of the prescription, users can adjust or switch off programs. Start with a small pilot group of experienced users and collect questionnaire feedback (GUIDE-008). Typical concern: "the device hears everything"; answer with on-device processing and transparent data sharing choices.

People & change

Involve management and the department lead (clinic or product owner) first, suppliers (device makers) and legal-compliance (MDR, liability) early, and the DPO before any fitting or usage data leaves the practice or the app: health data needs a clear legal basis and an AVV, and a DPIA is likely when data is reused for model training (GUIDE-005).

Works council: usually no. The devices serve customers or patients, not employee monitoring. It becomes relevant only if an employer rolls out hearables to staff and logs usage (§87(1) no. 6 BetrVG), then see GUIDE-004.

AI literacy: support staff and fitters with short training on what the noise reduction can and cannot do (Art. 4 AI Act, GUIDE-006). Communicate openly with users: the professional stays in control of the prescription, users can adjust or switch off programs. Start with a small pilot group of experienced users and collect questionnaire feedback (GUIDE-008). Typical concern: "the device hears everything"; answer with on-device processing and transparent data sharing choices.

Risks, regulation & compliance

  • Medical device: hearing aids are medical devices under Regulation (EU) 2017/745 (MDR), typically class IIa, requiring notified-body conformity assessment, clinical evaluation and post-market surveillance [7]. Software updates that change the algorithm may need re-evaluation.
  • EU AI Act: AI that is a safety component of, or itself, a product covered by MDR and requiring third-party conformity assessment is high-risk under Art. 6(1) with Annex I, with obligations applying from 2 August 2028 (deferred from August 2027 by the Digital Omnibus on AI, Reg. (EU) 2026/1744) [8]. Consumer earbuds without a medical claim are likely minimal risk. Not legal advice.
  • GDPR: audiograms, usage logs, location and sensor data from companion apps are health data (Art. 9); require explicit consent, data minimisation and EU processing; on-device processing of the audio itself is a privacy advantage. Avoid any speaker identification features that create voice biometrics unless they are strictly necessary and lawful (see SPEECH-010).
  • Safety and security: over-amplification (sound limits), safety of passing environment sounds (traffic, alarms), firmware update integrity and Bluetooth security (cyber requirements under MDR guidance and, for radio equipment, the RED).
  • Mitigations: audiological assessment before use, safe-output limits, transparent claims, real-world evidence collection with consent, clear user control of strength and an easy way to turn AI processing off.

Getting started

  1. For users: get an audiological examination first; trial 2-3 devices in your own noisy situations (restaurant, car, meeting) with a return window.
  2. For clinics: standardise speech-in-noise testing (e.g. Matrix or OLSA test) before and after fitting and log results per device generation.
  3. For developers: reproduce a Clarity Challenge baseline, measure STOI/HASPI and latency, then quantise a small model for an NPU.
  4. For manufacturers: map MDR class, AI Act Annex I applicability and notified-body timeline before locking the model architecture.
  5. Define a data policy: on-device audio only, pseudonymised telemetry, explicit consent for any model-improvement data.

Sources

  1. Sonova — announcement of Infinio platforms and first hearing aid with real-time AI — https://www.sonova.com/sonova-announces-two-new-hearing-aid-platforms-including-first-hearing-aid-real-time-ai-address/
  2. Phonak Audiology blog — Audéo Sphere: advancing speech understanding with AI — https://audiologyblog.phonakpro.com/audeo-sphere-advancing-speech-understanding-with-ai/
  3. HearingTracker — Phonak Audéo Infinio Ultra review — https://www.hearingtracker.com/hearing-aids/phonak-audeo-infinio
  4. Hearing Review — Phonak launches Audéo Sphere Infinio with real-time AI — https://hearingreview.com/hearing-products/hearing-aids/phonak-launches-infinio-hearing-aid-with-real-time-ai
  5. FDA final rule establishing OTC hearing aids (Federal Register, Aug 2022) — https://www.federalregister.gov/documents/2022/08/17/2022-17230/medical-devices-ear-nose-and-throat-devices-establishing-over-the-counter-hearing-aids
  6. Apple Newsroom — AirPods Pro 2 hearing aid feature (Sept 2024) — https://www.apple.com/newsroom/2024/09/apple-introduces-groundbreaking-health-features/
  7. Regulation (EU) 2017/745 (MDR) — https://eur-lex.europa.eu/eli/reg/2017/745/oj
  8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
  9. Clarity Challenge (hearing-aid speech enhancement) — https://claritychallenge.org/