Agricultural robots (weeding, harvesting)

Weeding and harvesting are labour-intensive, seasonal and increasingly hard to staff, while herbicide use is under regulatory and public pressure. Field robots combine real-time plant detection with a precise actuator (hoe blade, laser, single-nozzle spray, soft gripper) to treat individual plants instead of whole fields. Weeding robots are commercial today; selective harvesting is still limited to a few crops and high-value niches.

Problem & value

• Today: weeds are controlled with broad-spectrum herbicides, tractor-mounted hoes and hand weeding; fruit and vegetables are harvested by seasonal workers. Labour availability, wage costs, herbicide restrictions and the EU goal to reduce pesticide risk (Directive 2009/128/EC on sustainable use) push growers towards alternatives [4].
• AI value: per-plant decisions (crop vs. weed vs. ripe fruit) allow treatment only where needed, night and weekend operation, and a documented record of every action.
• Evidence: vendors such as Carbon Robotics claim up to 80 % lower weed-control cost and report hundreds of thousands of acres treated; these figures are vendor-stated and crop-dependent, so verify them in a trial on your own fields [1].

Variants

• Autonomous row-crop hoeing / seeding-weeding robots (sugar beet, vegetables): small solar or battery vehicles that work continuously, e.g. FarmDroid (DK, EU), Naïo Technologies (FR, EU), Agrointelli Robotti (DK, EU).
• Laser weeding: tractor-towed implements with many cameras and diode lasers (Carbon Robotics LaserWeeder, US), no herbicide and no soil disturbance [1].
• See-and-spray: camera-triggered nozzle arrays that spray only detected weeds (e.g. Bosch BASF Smart Farming "One Smart Spray", John Deere See & Spray) — reduce herbicide volume rather than replace it.
• Selective harvesting: strawberries, tomatoes, peppers, apples, asparagus, lettuce; mostly pilots and early commercial fleets, often sold as robots-as-a-service.
• Vineyards, orchards, greenhouses: pruning, thinning and de-leafing platforms.
• Private use: small garden weeding robots exist but are hobby products, outside this use case.

How it works

1. cameras + GNSS/IMU
2. image capture (strobed lighting for sunlight independence)
3. on-board detection/segmentation (crop, weed, fruit, ripeness, stem point)
4. localisation of target in 3D / machine frame, latency compensation
5. motion planning + actuator command (blade / laser dwell / nozzle / gripper)
6. safety layer (bumpers, LiDAR, person detection, geofence, e-stop)
7. logging, maps, fleet cloud, operator app

• Perception: CNN/transformer detectors and segmenters (YOLO-class, vision foundation models) running on embedded GPUs at tens of frames per second; harvesting adds depth sensing for 3D pose of fruit and peduncle.
• Control: classical navigation along rows (RTK-GNSS, row-following by vision); learned policies (imitation learning, RL in simulation) increasingly used for grasping and for leaf-occluded fruit.
• Human-in-the-loop: farmer defines field, crop and tolerance; remote supervisor or in-field operator handles stuck states; safety functions never depend only on the learned model.

Data preparation

The data is field images (weeds, crops, fruit) across varieties, soils, lighting and growth stages, plus field maps, GNSS tracks, application records and yield data. It lives on the robot, in the vendor portal, in farm-management software and often in spreadsheets or paper field books. Typical gaps are images from one season only, few hard negatives (weeds that look like crop, dust, shadows), no consistent field IDs, missing labels and unclear data ownership with the vendor.

Preparation steps: collect images across seasons and conditions, define classes before labelling (PREP-019), label with SAM-assisted tools and expert review (PREP-018), bring field records and spreadsheets into tidy tables (PREP-011) and get data out of vendor portals (PREP-005). Secure data export rights in the contract. For buyers, the preparation is mainly clean field boundaries, crop plans and records; for builders, collecting and labelling images is the largest cost item, often more than half of the project effort.

Implementation paths

Which path for whom

• SMB (family farms, growers): Path A, because proven weeding and harvesting products with dealer service exist for common crops and row spacings.
• Enterprise (large farms, producer groups, agri-business): Path B, because fleet data, weed maps and field records need to flow into farm-management systems, with own analytics on top of vendor machines.
• Developer (machine builders, agri-tech): Path C, because own actuators and uncovered crops need ROS 2 and open detectors; mechanics and field validation dominate the effort.

Path A — Buy (recommended for most SMB farms)

• Weeding: FarmDroid FD20, Naïo Orio/Ted/Oz, Agrointelli Robotti (all EU); Carbon Robotics LaserWeeder G2 (US, dealer/lease models); camera-based spraying from John Deere, Bosch BASF Smart Farming, Ecorobotix (CH, ultra-high-precision sprayer).
• Harvesting: Tevel (flying fruit-picking drones, IL), Advanced Farm (strawberries, US), Dogtooth (UK), Fieldwork Robotics (raspberries, UK), Octinion (BE, EU), Four Growers (tomatoes, US).
• Choose A when: crop and row spacing match a proven product; check dealer service within a day's reach, subscription vs. purchase, data ownership and export, and eligibility for EU/CAP investment subsidies.

Path B — Build on cloud APIs

• Little of the real-time loop belongs in the cloud (no reliable coverage, latency). Cloud APIs are useful for offline tasks: VLM-assisted labelling of field images, fleet analytics, agronomic advice from weed maps, and speech/chat interfaces for operators.
• Use EU-region services for storage and training pipelines; NVIDIA Isaac Sim/Omniverse for synthetic data.

Path C — Self-host (open source)

• Software: ROS 2 (Nav2, MoveIt 2, ros2_control), Open3D/PCL, OpenCV; detection with Ultralytics YOLO, RT-DETR, Segment Anything (SAM 2) for faster labelling; open datasets for weeds and fruit (e.g. CropAndWeed, WeedMap, PhenoBench).
• Hardware: HW1 on the machine (Jetson Orin NX/AGX class, TensorRT or ONNX Runtime); HW2/HW3 workstation for training. Research platforms: AgBot-class kits, Farmbot (open hardware).
• Choose C when: you are a machine builder, research institute or Agri-tech start-up needing own actuators or crops not covered by products. Effort is large: mechanics, safety certification and field validation dominate, not the model.

Path D — Custom training

Almost always needed at some level: fine-tune detectors on images from your crop varieties, soil colours, lighting and growth stages. Expect thousands to tens of thousands of labelled images per crop across seasons and regions; the Carbon Robotics "Large Plant Model" shows the vendor route of a foundation model over very large labelled plant data [1]. Collect hard negatives (weeds that look like crop, dust, shadows, residue) and test across seasons, as models degrade with domain shift.

State of the art (2026)

• Weeding is commercial: autonomous hoeing robots and laser weeders are in routine use, with dozens of EU deployments and a growing dealer ecosystem; see-and-spray is now offered by large OEMs.
• Foundation models: vendors train large plant models and use VLMs to label data and handle new species with few examples; edge inference on embedded GPUs reaches the throughput needed for continuous driving [1].
• Harvesting: pick rates, damage rates and cost per kg remain the limit; delicate crops with occlusion (strawberries, peppers) are first, apples and tomatoes follow. Learned grasping and VLA-style policies (see ROBO-004) are research-to-pilot stage.
• Limits and failure modes: strong sunlight, dust, mud and wet leaves; crop and weed at the same growth stage; out-of-distribution species; GNSS loss under trees; slow battery recharge or low speed for small machines; maintenance of lasers, nozzles and grippers; theft of RTK equipment. A false positive can destroy crop, so confidence thresholds and "crop-safe" defaults are essential.

Shared resources & combinations

• Shared stack: object-detection + segmentation + edge-runtime with ROBO-001 (picking), ROBO-011 (TinyML), ROBO-016 (person detection safety) and ROBO-002 (navigation); simulation and sim-to-real tooling with ROBO-010.
• Data pipeline: images from field robots feed VIS-005 (crop disease), PRED-013 (yield prediction) and OPT-014 (precision farming), and drone maps from ROBO-006 / VIS-003 provide treatment plans.
• Livestock side: VIS-014 and PRED-044 for barns and pastures.

Effort, cost & KPIs

• Path A: weeks to deploy (field mapping, training the operator, one season of tuning); typical cost drivers are purchase or lease, RTK service, spare parts, electricity or fuel, and service contract.
• Path C/D: 12–36 months to a field-ready machine incl. safety and CE conformity.
• KPIs: weed kill rate and crop damage rate (per field and per pass), area per hour, herbicide volume or hand-weeding hours saved, cost per hectare, uptime/interventions per day, pick rate (kg/h), damage and miss rate for harvesting, share of ripe fruit left behind.

People & change

Management (owner) decides on investment and subsidies. The department lead (farm or field manager) and the machine operators work with the robot daily and must be part of selection and field tests from the start; they know the soil and crop conditions that decide success. IT or a service partner handles connectivity and accounts; suppliers (dealer, vendor) are involved for service levels and data ownership.

Works council: depends. Many farms have none; where one exists, robot telemetry that records operator logins or working times can be suited to monitor behaviour (§87(1) no. 6 BetrVG), and §90(1) no. 3 requires early information about AI-planned work processes (GUIDE-004). Seasonal workers should be informed about cameras in the fields.

Data protection: mostly no personal data, but cameras can capture people and farm data ties to individuals; ask the DPO if staff or neighbours may be filmed (GUIDE-005). AI literacy: support staff with hands-on training and a named robot champion (GUIDE-006, GUIDE-008). Typical concerns are job loss, breakdown in harvest peak and safety; address them with a pilot on one field, a service agreement, safety zones and operators who can stop the machine at any time.

Risks, regulation & compliance

• Machinery Regulation (EU) 2023/1230: applies from 20 January 2027 and replaces the Machinery Directive; machines with safety functions using self-evolving, learning behaviour fall into the higher-risk annex requiring third-party conformity assessment. Autonomous agricultural machines also follow ISO 18497 (safety of highly automated agricultural machines) and ISO 13849 for safety-related control [2].
• EU AI Act (Reg. 2024/1689): a safety component of a product covered by the Machinery legislation can be high-risk (Art. 6(1), Annex I) (Annex I high-risk obligations apply from 2 Aug 2028 after the Digital Omnibus, Reg. (EU) 2026/1744); a pure agronomic detector without a safety function is typically minimal risk. Classification depends on whether the AI performs a safety function (e.g. person detection to stop the machine) [3].
• No Art. 5 issue in normal use; do not use cameras to monitor or score workers' behaviour (workplace emotion recognition is prohibited).
• GDPR: field cameras can capture workers, passers-by, vehicles and neighbours' land; minimise and blur, define retention, inform staff and works council. Farm data ownership and sharing is also governed by the EU Data Act (Reg. 2023/2854), which gives users of connected machines access to the data generated [5].
• Plant protection: sprayer robots must comply with pesticide rules and application records; laser or mechanical weeding may qualify for organic production, check with the certifier.
• Operational: liability when a robot damages crops, people or wildlife (hares, ground-nesting birds), road use of towed implements, laser safety class (eye protection, enclosures), cyber security of fleet cloud connections.

Getting started

1. Define the crop, row spacing and the most expensive operation (hand weeding, herbicide passes).
2. Visit a demo farm and request a reference list of same-crop users in your region; ask for independent trial data, not just vendor claims.
3. Run a one-field, one-season pilot with clear KPIs (kill rate, crop loss, hours saved).
4. Check subsidy options and insurance, and ensure RTK correction and mobile coverage.
5. Capture images and logs from day one; they are the base for later own model tuning.

Sources

1. GeekWire — Carbon Robotics releases new line of laser weed zappers (LaserWeeder G2) — https://www.geekwire.com/2025/carbon-robotics-releases-new-line-of-laser-weed-zappers-for-more-types-of-farms-and-budgets/
2. Regulation (EU) 2023/1230 on machinery — https://eur-lex.europa.eu/eli/reg/2023/1230/oj
3. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
4. Directive 2009/128/EC on the sustainable use of pesticides — https://eur-lex.europa.eu/eli/dir/2009/128/oj
5. Regulation (EU) 2023/2854 (Data Act) — https://eur-lex.europa.eu/eli/reg/2023/2854/oj
6. FarmDroid — https://farmdroid.com
7. Naïo Technologies — https://www.naio-technologies.com
8. ROS 2 documentation — https://docs.ros.org