Bioacoustics & wildlife sound monitoring

Ecologists used to count birds, bats or whales by ear or by hand-labelling spectrograms, which limits surveys to a few hours per site. Autonomous recorders now capture months of audio, and neural classifiers (BirdNET, Perch and similar) turn it into species lists, activity curves and alerts. The human role shifts from listening to validating detections and designing the survey.

Problem & value

• Today: point counts by experts, short manual listening sessions, or semi-manual review of bat call files. Coverage is small, observer-dependent and expensive; night-time and remote sites are barely covered.
• AI value: passive acoustic monitoring (PAM) records continuously and non-invasively; classifiers reduce thousands of hours of audio to a ranked list of candidate detections that experts verify. This enables long time series for conservation status reporting, impact assessments (wind farms, construction) and restoration success checks.
• Evidence: BirdNET covers thousands of species and is widely used in research and citizen science; Google's Perch models and the BEANS/BirdSet benchmarks exist because the field needs comparable evaluation [1][2][3]. Published figures vary strongly by species, region and noise, so measure precision and recall on your own site rather than relying on vendor numbers.

Variants

• Terrestrial birds and amphibians: the most mature case (BirdNET, Perch, Kaleidoscope).
• Bats: ultrasonic detectors plus call classifiers (e.g. BatDetect2, Wildlife Acoustics Kaleidoscope, Elekon Batlogger); used for mandatory bat surveys in planning.
• Marine / underwater: hydrophones for whales, dolphins (including harbour porpoise by-catch and shipping noise studies), fish choruses.
• Insects and soundscape ecology: acoustic indices (ACI, NDSI) and insect decline monitoring.
• Operational protection: wind-turbine curtailment on bat or bird activity (see VIS-060), poaching or chainsaw detection in reserves (generic audio events, see SPEECH-009).
• Private / citizen science: smartphone apps (BirdNET, Merlin Sound ID) for hobby birders and community projects.

How it works

1. recorders (AudioMoth, Song Meter, hydrophone)
2. storage/upload (SD, LoRa/4G summaries)
3. preprocessing (resample, segment 3-5 s, spectrogram/mel)
4. classifier or embedding model (species scores per window)
5. post-processing (thresholds per species, location/date species filter, smoothing)
6. expert validation of a sampled subset (human-in-the-loop)
7. statistics (occupancy, activity, richness)
8. report / dashboard / alert

• Classifier: CNN or transformer on mel spectrograms trained on labelled recordings (xeno-canto, Macaulay Library, Xeno-canto-derived BirdSet). Output is a score per species and window.
• Embeddings + few-shot: foundation models such as Perch produce embeddings; a small linear head or nearest-neighbour search with a vector index finds new species or call types from a handful of examples (agile modelling).
• Edge variant: a small model on the recorder (Raspberry Pi, Jetson, microcontroller) filters or classifies on site and sends only detections, saving storage and power.
• Human-in-the-loop: experts verify a stratified sample of detections per species to estimate precision; thresholds are set per species; rare or protected species are always manually confirmed before they enter a legal report.

Data preparation

The data is field audio from autonomous recorders (SD cards, hydrophones, ultrasonic bat detectors) plus site metadata such as GPS, date and habitat. In practice it sits on SD cards, laptops, external drives and project file shares, with metadata in spreadsheets. Typical gaps are inconsistent file names, missing or wrong timestamps and coordinates, wind and rain noise, duplicate copies, and very few expert-verified labels for local species.

Steps: 1. Inventory recordings and metadata (PREP-001). 2. Standardise file naming, time zones and metadata in a tidy table (PREP-011). 3. Define the species list and label scheme (PREP-019). 4. Have ornithologists or bat experts verify a sample as a test set (PREP-018). 5. Archive raw audio with a backup and storage plan; use ffmpeg and sox for conversion.

Pre-trained detectors need no training data, but local validation is essential. Preparation is roughly 30 to 40 percent of the effort, mainly metadata cleaning and expert verification.

Implementation paths

Which path for whom

• enterprise: Path C, because BirdNET, Perch and bat classifiers run on HW1 and the same pipeline serves many sites and projects.
• developer: Path C, because open models and datasets are the foundation for custom detectors.
• public: Path C, because authorities and research bodies need transparent, reproducible results and data sovereignty for protected-species data.
• private: Path A, because smartphone apps such as BirdNET are mature and free.

Path A: Buy (off-the-shelf)

• Recorders and analysis software: Wildlife Acoustics (Song Meter, Kaleidoscope Pro), Audiomoth ecosystem (Open Acoustic Devices, UK), Elekon (CH, Batlogger) bat detectors, Titley Scientific, Bioacoustica-style SaaS.
• Platforms: Arbimon (Rainforest Connection), Cornell's BirdNET apps, Wildlife Insights is image-based (see VIS-032). EU-based options include BirdNET-derived tooling hosted by European institutes and Naturalis-type research infrastructures; check hosting location.
• Choose A when: a consultancy or authority needs standard surveys with little ML expertise.

Path B: Build on cloud APIs

• No dominant general ASR-style API exists for animal sound. Typical pattern: store audio in EU cloud storage, run BirdNET or Perch models in a container on EU compute (AWS Frankfurt, Azure West Europe, OVHcloud, Scaleway, STACKIT), results into a database, dashboards in Grafana or Streamlit.
• General audio embedding APIs are possible but open bioacoustic models usually perform better.

Path C: Self-host (open source), the usual choice

• Models and tools: BirdNET-Analyzer (Cornell Lab / Chemnitz University of Technology, Germany, open source, GUI and CLI), Perch 2.0 (Google DeepMind, open weights), BatDetect2 for bats, BirdSet datasets and benchmark, OpenSoundscape (Python), PAMGuard and Ocean Sound tools for marine mammals, AVES / BEATs / BioLingual style audio encoders.
• Stack: Python, PyTorch or TensorFlow Lite, ffmpeg, a vector index (FAISS, pgvector), a database with site metadata.
• Hardware: HW1 handles inference (BirdNET runs on a Raspberry Pi in near real time); HW2 speeds up bulk processing of terabytes and fine-tuning.
• Ops effort: low to medium; the real effort is data logistics (SD card collection, storage, naming) and validation.

Path D: Custom training

Worth it for taxa or regions poorly covered by public models (local dialects, rare species, tropical or underwater sounds). Requires hundreds of verified clips per target class plus realistic background negatives from your own sites; fine-tune a pretrained encoder rather than training from scratch. Domain shift between recorder models and habitats is the main reason for retraining.

State of the art (2026)

• Foundation bioacoustic models: Perch 2.0 and similar encoders trained on large multi-taxa corpora give strong transfer and few-shot performance; BirdNET v2.x remains the most deployed bird classifier [1][2].
• Benchmarks: BirdSet and BEANS standardise evaluation; results show a large gap between curated focal recordings and real soundscapes with overlapping calls [3].
• Trends: on-device inference, active learning to prioritise annotation, multi-label detection of overlapping species, and integration with camera traps (VIS-032) for multimodal biodiversity monitoring.
• Limits and failure modes: false positives from wind, rain, insects and human voices; poor recall for quiet or rare species; classifier scores are not calibrated probabilities; regional species lists matter; no model gives abundance (individual counts) reliably; recorder microphone differences shift performance.

Shared resources & combinations

• Same building blocks (audio-classification, edge-runtime): SPEECH-009 acoustic event detection, SPEECH-008 audio denoising, ROBO-011 TinyML sensors.
• Combinations: VIS-032 (camera-trap species ID) for multimodal surveys, VIS-060 (bird collision avoidance at wind turbines), VIS-013 (environmental monitoring). A shared embedding index and edge fleet management can serve all of these.

Effort, cost & KPIs

• Path A: weeks; recorder hardware is the main cost (tens to a few hundred euros per unit for AudioMoth-class devices, more for professional detectors; check current prices).
• Path B/C: 1-3 months for a validated pipeline; storage grows quickly (continuous recording produces terabytes per season).
• Path D: 3-6 months including annotation.
• KPIs: per-species precision and recall on a validated sample, detections per recorder-night, share of audio reviewed by humans, species richness trend, recorder uptime, cost per surveyed site-night.

People & change

Involve management or the project owner (budget, scope), the department lead for ecology or nature conservation, field staff and ecologists as end users, and IT for storage and compute. Data owners such as land owners, authorities and research partners must agree on data sharing. Legal or compliance clarifies permits for protected species and publication of locations.

Works council: usually no, because the system analyses animal sounds, not staff (§87(1) no. 6 BetrVG); inform it if field work processes change (§90(1) no. 3). In public bodies, tell the staff council. Data protection: recordings can capture human voices; configure recorders and filters to discard speech and involve the DPO, with a DPIA only if speech cannot be excluded at scale.

AI literacy: support ecologists with training on confidence scores, false positives and how to verify detections, in line with AI Act Art. 4. Start with a pilot at a few sites and compare against manual surveys. Humans stay in control: only expert-verified detections go into official reports. A typical concern is trust in automated species lists; address it by publishing precision and recall from your own validation, and involve citizen-science volunteers openly.

Guides: GUIDE-009, GUIDE-003, GUIDE-006.

Risks, regulation & compliance

• EU AI Act: minimal risk for wildlife analysis (no Annex III category). AI literacy (Art. 4) applies. Reusing the same audio pipeline to identify people (speaker identification) or for surveillance would change the classification; that is outside this use case (see SPEECH-010, and note the Art. 5 limits on biometric identification).
• GDPR: recorders in forests, parks or near paths can capture human speech. Mitigate with placement, ultrasonic or low-sample-rate recording, voice-activity detection that discards or masks speech on device, short retention and a documented DPIA where people may be recorded. Recordings of identifiable conversations are personal data.
• Nature law: results often feed legal processes (EU Birds and Habitats Directives, Natura 2000, environmental impact assessments). Authorities may require expert-verified detections and documented methods, so AI output is evidence support, not a final determination. The EU Nature Restoration Regulation (2024/1991) increases monitoring demand [4].
• Data governance: location data for rare species can enable poaching; restrict access and obscure coordinates in public datasets. Respect licences of training data (xeno-canto Creative Commons terms).
• Scientific risk: unvalidated detections inflating species lists; mitigate with per-species thresholds and sample verification.

Getting started

1. Define the question (presence, activity trend, richness) and the target taxa, then choose recorder type and schedule (for example dawn chorus windows).
2. Run BirdNET-Analyzer or Perch on a pilot set of 1-2 weeks of recordings from a few sites.
3. Have an expert label a random sample per species to compute precision and set per-species thresholds.
4. Add privacy controls (speech filtering, retention policy) before wider deployment.
5. Automate ingest, inference and a dashboard; consider fine-tuning only after seeing where the public models fail.

Sources

1. BirdNET-Analyzer (Cornell Lab of Ornithology / Chemnitz University of Technology) - https://github.com/birdnet-team/BirdNET-Analyzer
2. Google Research Perch (bioacoustics models) - https://github.com/google-research/perch
3. Rauch et al., BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics - https://arxiv.org/abs/2403.10380
4. Regulation (EU) 2024/1991 on nature restoration - https://eur-lex.europa.eu/eli/reg/2024/1991/oj
5. OpenSoundscape - https://github.com/kitzeslab/opensoundscape
6. BatDetect2 - https://github.com/macaodha/batdetect2
7. AudioMoth, Open Acoustic Devices - https://www.openacousticdevices.info/
8. Regulation (EU) 2024/1689 (AI Act) - https://eur-lex.europa.eu/eli/reg/2024/1689/oj