Humanoid & general-purpose robots (vision-language-action models)

Classic robots are programmed for one task in one cell. Vision-language-action (VLA) models map camera images and a text instruction directly to robot actions, so one model can be re-tasked ("put the tote on the shelf") without re-programming. Combined with humanoid or other mobile-manipulator hardware, this promises robots for unstructured work. In 2026 this is real in pilots, not yet a mainstream purchase: reliability, speed, safety certification and cost per task are the open questions.

Problem & value

• Today: automation is task-specific. Each new product, pallet or workstation needs integrators, fixtures and programming (see ROBO-001, ROBO-003). Labour-intensive, variable tasks (tote handling, kitting, machine tending, restocking, lab work) stay manual, and many EU employers report difficulty filling such jobs.
• AI value: a pretrained generalist policy can be adapted with tens to hundreds of demonstrations instead of months of engineering, understands language instructions and generalises somewhat to new objects and scenes. Humanoid form factors aim to work in spaces built for people without rebuilding the facility.
• Evidence status: published results are mostly lab benchmarks and vendor demos. Treat throughput, uptime and cost claims as unverified until measured in your own pilot. No independent, large-scale ROI data for humanoids exists yet.

Variants

• Humanoid for logistics: tote and case handling in warehouses (e.g. Agility Robotics Digit, Apptronik Apollo, Figure) in fenced or segregated zones.
• Wheeled mobile manipulator: arm on an AMR base (cheaper, more stable than legged), the pragmatic choice for most pilots (see ROBO-002, ROBO-008).
• Industrial arm with a VLA "brain": fixed arm adding language and generalisation to ROBO-001/003 workcells.
• Research and education platforms: low-cost arms and open hardware (SO-100/SO-101 arms, Koch arm, Unitree humanoids) for universities and developers.
• Service and care: reception, logistics in hospitals, elder-care assistance. Early research, with strong ethical and safety constraints.
• Private use: hobbyist arms and home-robot prototypes; no mature consumer product for unsupervised household work.

How it works

1. instruction (text/voice) + cameras + proprioception
2. VLA model (vision-language backbone + action head: diffusion / flow matching / tokens)
3. action chunk (next 0.5-2 s of joint targets or end-effector poses)
4. low-level controller (impedance/whole-body control, trained RL locomotion policy)
5. safety layer (certified, non-AI: speed/force limits, protective stops, geofences)
6. execution
7. new observation logging
8. teleop correction & fine-tuning
9. evaluation in simulation
10. staged rollout

• VLA models: a pretrained vision-language model is extended with an action output. Examples: RT-2 and Gemini Robotics (Google DeepMind), pi0 / pi0.5 (Physical Intelligence), OpenVLA, NVIDIA Isaac GR00T N1.x, SmolVLA (Hugging Face). Many use a dual-system design: a slow "reasoning" model plans, a fast action model executes.
• Whole-body control: locomotion and balance of legged robots are mostly handled by RL-trained policies (trained in simulation, see ROBO-010) rather than the VLA itself.
• Data: teleoperation demonstrations, human video, simulation and synthetic data. Data collection is the main cost.
• Human-in-the-loop: operator-defined task scope, remote assistance for stuck robots, teleoperation fallback, mandatory certified safety functions that do not depend on the neural network.

Data preparation

Needed: task definitions and standard operating procedures, site maps and station layouts, teleoperation demonstration data, simulation assets (CAD of workplaces, tools, totes) and safety documentation. In a company these sit in work instructions on file shares or paper, in CAD/PDM, in WMS/MES and in safety files of the EHS team. Typical gaps: unwritten tacit procedures, outdated layouts, no recorded demonstrations, inconsistent item data, and unclear rights to record and reuse video that shows employees.

Steps: (1) select two or three narrow tasks and write precise procedures with the experts (PREP-016, PREP-014); (2) digitise instructions and layouts (PREP-006) and clean item master data (PREP-005, PREP-011); (3) build a digital twin or simulation of the workplace; (4) collect teleoperated demonstrations with outcome labels (PREP-019, PREP-020); (5) define acceptance test scenarios (PREP-018); (6) clarify the legal basis and anonymisation for any recordings of people (PREP-025, PREP-022). Tools: vendor fleet platform, simulators, CVAT. Data effort is high, often 40 percent or more of the project, because embodied data barely exists in companies. See GUIDE-003.

Implementation paths

Which path for whom

• Enterprise: Path A. Humanoids are early-stage; a vendor pilot with the manufacturer's models and support limits risk and avoids building embodied AI in-house.
• Developer: Path C. Open VLA models and simulators (for example open-weight VLA checkpoints, Isaac Sim, MuJoCo) on HW2/HW3 are the practical basis for research and products.
• Public: Path C. Publicly funded projects and research need data sovereignty and reproducibility, so open models on own or sovereign infrastructure fit best.

Path A — Buy (recommended: pilot with a vendor)

• Humanoids and general-purpose platforms: Agility Robotics (Digit), Apptronik (Apollo), Figure AI, Tesla (Optimus, not commercially available), Boston Dynamics (Atlas, Spot), Unitree (low-cost, China), UBTech (China). EU: NEURA Robotics (Germany, 4NE1 humanoid, MAiRA), Humanoid/Sanctuary-type vendors are mostly non-EU; also PAL Robotics (Spain, TIAGo, Kangaroo), Wandelbots (Germany, no-code programming, ROBO-003).
• Robot foundation model providers: Physical Intelligence (pi0 family), Google DeepMind (Gemini Robotics, via trusted-tester access), NVIDIA (GR00T, Isaac, Jetson Thor).
• Choose A when: you have a defined, repetitive but variable task, a safe zone for the pilot, and can accept vendor lock-in and RaaS (robots-as-a-service) pricing. For fixed tasks, conventional robots (ROBO-001) are still more reliable and cheaper.

Path B — Build on cloud APIs

• Use a frontier VLM (Claude, Gemini, GPT, Mistral) as a high-level planner, scene describer or task verifier that calls a skill library (grasp, place, navigate) via tool calling. Gemini Robotics-ER offers embodied reasoning (pointing, spatial understanding, task planning) via the Gemini API.
• Latency (hundreds of ms to seconds) and network dependence forbid cloud models in the real-time control loop; keep control on-board.
• EU data residency: use EU regions; camera streams may show employees, so minimise upload (send frames only on demand).

Path C — Self-host (open source)

• Open models: OpenVLA (Stanford/Berkeley, 7B), openpi (pi0, pi0-FAST, pi0.5 weights from Physical Intelligence), NVIDIA GR00T N1.x (open weights for research and commercial use under NVIDIA licence), SmolVLA (~450M parameters, Hugging Face LeRobot), Octo. Check licence terms of each (some are non-commercial or have use restrictions).
• Stack: LeRobot (Hugging Face) for data, training and deployment; ROS 2 for integration; MuJoCo, NVIDIA Isaac Sim/Lab, Genesis for simulation.
• Hardware: HW1 for small models on-board (Jetson Orin/Thor class); HW3 for fine-tuning 3-7B VLAs; HW4 for larger training runs. Low-cost arms (around a thousand euros) make a research setup affordable.

Path D — Custom training

Fine-tuning a pretrained VLA on your own embodiment and task is the standard way to reach usable success rates. Needs tens to a few hundred teleoperated demonstrations per task, a calibrated camera setup and an evaluation protocol (success rate over many repeated trials). Training from scratch is only for labs with fleets and large data.

State of the art (2026)

• Generalist policies (pi0.5, Gemini Robotics 1.5, GR00T N1.x) show open-world generalisation in homes and kitchens, cloth folding, assembly and multi-step tasks, with dual "think then act" architectures.
• Cross-embodiment training (Open X-Embodiment dataset, 20+ institutions) lets one model drive different robots.
• Data scaling: teleoperation, UMI-style handheld grippers, egocentric human video and simulation are replacing pure lab collection.
• Hardware: commercial humanoids run first paid pilots in logistics (tote moving) but at modest speed and with limited runtime per battery charge; most videos are teleoperated or scripted, so ask for unedited autonomous runs.
• Limits and failure modes: success rates on new tasks are often well below industrial requirements; fragile at unseen lighting/objects; slow; poor force-sensitive and dexterous work; hallucinated or unsafe actions are possible and not formally verifiable; benchmarks (LIBERO, SimplerEnv, RoboArena-style real-world evaluation) do not predict plant performance.

Shared resources & combinations

• Simulation and sim-to-real: ROBO-010 (Isaac Sim/Lab, MuJoCo), the same pipeline trains locomotion and tests policies.
• Perception stack: ROBO-001 (grasping, 3d-vision, segmentation), ROBO-003 (demonstration and language programming).
• Mobility and fleets: ROBO-002 (AMR navigation, fleet management), ROBO-015 (inspection rounds), ROBO-014 (unloading).
• Safety: ROBO-016 (AI-based safety functions, speed and separation monitoring).
• Service settings: ROBO-008.
• Hardware: on-board edge GPUs (Jetson class) shared with other edge vision use cases; HW3 server shared for fine-tuning with VLM use cases.

Effort, cost & KPIs

• Path A pilot: 3-6 months incl. site safety assessment; hardware or RaaS cost is high (humanoids are typically six-figure EUR per unit; low-cost research humanoids far less), plus integration.
• Path C research setup: weeks to first demo on a low-cost arm; months for a robust task.
• Cost drivers: hardware, safety engineering and certification, data collection, remote operators, maintenance.
• KPIs: autonomous task success rate (n >= 50 trials), picks or totes per hour vs. human baseline, intervention rate per hour, mean time between failures, uptime per charge, safety incidents and near misses, cost per task.

People & change

Involve management early because pilots are costly and visible, the department lead as process owner, IT security for networked robots with cameras, legal and compliance for machinery safety, product liability and the AI Act, HR for roles and qualification, and above all the staff who will work next to the robot (GUIDE-001). Works council: yes. A humanoid carries cameras and sensors, logs movements and interactions in the workplace and changes work processes, so it is objectively suited to monitor behaviour or performance (§87(1) no. 6 BetrVG, BAG); add §90(1) no. 3 and the expert right under §80(3) for AI. Public sector: involve the Personalrat. Negotiate a works agreement on sensor data, zones, retention and no performance evaluation (GUIDE-004). A DPO check and likely a DPIA are needed if video of people is processed (GUIDE-005). AI literacy: the AI Act requires measures to support staff literacy, so train colleagues on safe interaction, limits and emergency stops (GUIDE-006). Typical concerns are safety, replacement fears and hype. Communicate honestly that today humanoids handle narrow tasks, start with a small, voluntary pilot with AI champions (GUIDE-008, GUIDE-009), keep humans able to stop and override the robot at any time and define go/no-go criteria upfront.

Risks, regulation & compliance

• EU AI Act: classification depends on use. A robot is typically a machine whose safety component can be high-risk under Art. 6(1) / Annex I (Machinery Regulation (EU) 2023/1230, applicable from 20 January 2027) if the AI is a safety component subject to third-party conformity assessment; high-risk obligations for Annex I products apply later than Annex III (2 Aug 2028, postponed from Aug 2027 by the Digital Omnibus on AI, Reg. (EU) 2026/1744). Non-safety uses are minimal risk. Art. 5 prohibitions: emotion recognition of workers (workplace) and manipulative or exploitative techniques are banned; a humanoid in a care or reception role must not infer employees' or patients' emotions. Art. 50 transparency applies when a robot interacts with people (disclose it is an AI/robot).
• Machinery Regulation 2023/1230 and standards: self-evolving behaviour and AI safety functions require risk assessment over the lifecycle; ISO 10218 (industrial robots), ISO/TS 15066 (collaborative), ISO 13482 (personal care robots) and ISO 25785-1 (legged and dynamically stable robots, in development) are the relevant norms. A learned policy alone cannot be the safety function.
• GDPR: onboard cameras and microphones capture workers, customers and patients. DPIA, signage, on-device processing, short retention, works council co-determination (Germany: BetrVG §87).
• Liability: revised Product Liability Directive (EU) 2024/2853 covers software and AI; manufacturers and deployers share responsibility.
• Cybersecurity: Cyber Resilience Act (EU) 2024/2847 for products with digital elements; secure remote access, signed updates.
• Other risks: physical injury (heavy, fast, tipping robots), workforce acceptance, vendor lock-in, vendor viability, demo-vs-reality gap.
• Mitigations: separate safety zones, certified safety controller, speed and force limits, remote kill switch, staged autonomy, independent acceptance tests.

Getting started

1. Pick one narrow task (e.g. tote transfer between two stations) and measure the human baseline.
2. Do a hazard and risk assessment first; define a fenced or segregated pilot zone with certified safety functions.
3. Ask 2-3 vendors for unedited autonomous runs on your objects; agree success-rate, intervention and uptime criteria.
4. In parallel, build a low-cost lab setup (LeRobot with an SO-101 arm or similar) to learn data collection and fine-tuning.
5. Involve the works council and data protection officer early; run a 3-month pilot with weekly KPI review before scaling.

Sources

1. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
2. Regulation (EU) 2023/1230 (Machinery Regulation) — https://eur-lex.europa.eu/eli/reg/2023/1230/oj
3. Physical Intelligence — openpi (pi0, pi0.5) — https://github.com/Physical-Intelligence/openpi
4. NVIDIA Isaac GR00T — https://github.com/NVIDIA/Isaac-GR00T
5. OpenVLA — https://openvla.github.io/
6. Hugging Face LeRobot / SmolVLA — https://github.com/huggingface/lerobot
7. Google DeepMind — Gemini Robotics — https://deepmind.google/models/gemini-robotics/
8. Open X-Embodiment collaboration — https://robotics-transformer-x.github.io/
9. Directive (EU) 2024/2853 (Product Liability) — https://eur-lex.europa.eu/eli/dir/2024/2853/oj
10. Regulation (EU) 2024/2847 (Cyber Resilience Act) — https://eur-lex.europa.eu/eli/reg/2024/2847/oj