Photo/video-to-3D capture (Gaussian splatting, NeRF) for tours, products & digital twins

Classic 3D scanning needs laser scanners or hours of photogrammetry clean-up, and plain photos or 360 panoramas cannot be walked through. Neural reconstruction (3D Gaussian Splatting, 3DGS, and Neural Radiance Fields, NeRF) turns an ordinary phone or drone video into a photorealistic 3D scene that a viewer can orbit in a browser, with reflections and fine detail such as foliage that meshes handle poorly. Typical uses are property and venue tours, 3D product views, heritage documentation and a visual layer for digital twins.

Note: this file was written without live web research (search quota exhausted); product and version statements reflect knowledge up to mid-2026 and should be re-verified before decisions.

Problem & value

• Today: real-estate, hospitality and venue marketing relies on photos, 360 tours (e.g. Matterport-style capture with dedicated cameras) or hand-modelled 3D. E-commerce 3D views need studio turntables or artists. Heritage and as-built documentation use terrestrial laser scanning or photogrammetry, which are accurate but costly and slow to process.
• AI value: capture with a phone in minutes; training takes minutes to an hour on one GPU or in a cloud service; the result renders in real time (often well above 30 fps) in browsers and on phones. Gaussian splatting made real-time rendering of NeRF-quality scenes practical [1].
• Production evidence: Zillow launched SkyTour in July 2025, a Gaussian-splatting 3D exterior tour built from drone footage on eligible Showcase listings (Zillow press release, 15 Jul 2025); Polycam, Luma AI and Niantic Spatial (Scaniverse) offer consumer and professional splat capture; Google and others use neural reconstruction in mapping and immersive-view features. Quantified business uplift is vendor-reported and not independently established, so measure it yourself.
• Not a measurement tool by default: splats look right but are not guaranteed metrically accurate; survey-grade work still needs control points or LiDAR.

Variants

• Real estate & hospitality: walkthrough of apartments, hotels, event venues, often combined with floor-plan data (GEN-018, GEN-009).
• E-commerce / product: object-centric capture on a turntable, converted to splat or mesh for AR and 3D viewers (GEN-008).
• Cultural heritage & museums: documentation of monuments, objects and excavations; public-sector archives.
• Construction & as-built: periodic site capture for progress comparison, reality capture against BIM (ROBO-009, VIS-021).
• Digital twins & simulation: photoreal environment layer for facility twins and robot simulation (ROBO-010).
• Media & VFX: location capture for film and games (GEN-021).
• Private use: scanning a favourite place, a sculpture or a collection item with a phone app.

How it works

1. capture (phone/drone/360 video, optional LiDAR)
2. frame selection, blur filtering, optional masking of people/sky
3. camera pose estimation (COLMAP / SfM, or on-device ARKit/ARCore poses)
4. optimisation: initial sparse point cloud
5. train millions of 3D Gaussians (or NeRF)
6. cleanup: crop floaters, remove background, align/scale, optional mesh extraction
7. compress + export (.ply, .spz, .sog, glTF Gaussian-splat extension)
8. web viewer / AR / engine plugin / twin platform

• Gaussian splatting represents a scene as many anisotropic coloured 3D blobs, rasterised in real time; training is gradient-based from posed images [1]. NeRF uses a neural network queried along rays: smaller files but slow rendering, so now mostly used in research and as a baseline [2].
• Feed-forward reconstruction (models predicting geometry and poses without per-scene optimisation, e.g. DUSt3R/VGGT-style models) is shrinking the pipeline from minutes to seconds and removes the separate SfM step in some tools.
• Human-in-the-loop: capture guidance (coverage check), operator cleanup of floaters and privacy masking, approval before publication; for measured deliverables, survey checks.

Data preparation

The input data is captured on purpose rather than extracted from existing systems: phone, drone or 360 video, optional LiDAR. The existing assets are listing photos in a CMS or portal, floor plans in a DMS, product images in a PIM or shop system, and survey data on file shares. Typical gaps are blurry or insufficiently overlapping footage, reflective or textureless surfaces, moving people and number plates in frame, missing consent from residents or venue owners, unlabelled scans without location or scale, and scattered storage on private phones.

Preparation steps: - Write a capture protocol (60-300 images per object, slow walking path, constant exposure, no moving objects, scale reference) and check coverage on site. - Mask or remove faces, plates, documents and personal items before upload or publication. - Store raw footage plus results in a structured folder or DAM with metadata for object, date, location and scale; see PREP-010 and PREP-009. - Keep an open export (.ply, glTF) next to the vendor link to avoid lock-in. - For personal data in scenes use PREP-022 and PREP-024.

Rough share of effort: about 10-20 percent, mostly capture discipline and privacy masking. Path C adds pose-estimation tuning.

Implementation paths

Which path for whom

• SMB: Path A, because Polycam, Luma, Scaniverse or Postshot turn phone footage into shareable scenes within hours and the process is standard.
• Enterprise: Path B, because listing platforms, product catalogues or twin tools need automated, multi-tenant pipelines with EU-region GPU batch jobs.
• Developer: Path C, because Nerfstudio, gsplat and COLMAP give full control, avoid format lock-in and run on a single HW2 GPU.
• Public: Path C, because heritage, building and infrastructure scans should stay on sovereign or in-house infrastructure and be archived in open formats (glTF splat extension).
• Private: Path A, because free phone apps with on-device processing (Scaniverse) are enough for hobby, travel and family use.

Path A — Buy (recommended for most SMBs)

• Capture-and-host apps: Polycam, Luma AI, Niantic Scaniverse (free phone app, on-device processing), KIRI Engine, Postshot (Jawset, desktop training, Germany-based, EU), Teleport.
• Real-estate and venue platforms: Matterport and similar tour vendors add or experiment with splat-based views; check current feature status. Reality-capture suites such as RealityScan (Epic) and Pix4D (Switzerland) handle survey-style photogrammetry and increasingly splat outputs.
• Web viewers/hosting: SuperSplat (PlayCanvas, open-source editor and viewer), Spline and others; self-hosted static files are possible since splats are plain files.
• Choose A when: under a few hundred scenes per month, no strict metric accuracy needs, publishing via share links or iframes is acceptable. Check: export formats (avoid lock-in), data residency, licence for commercial use of scans, retention of your uploads.

Path B — Build on cloud APIs

• Hosted reconstruction APIs: Luma and Polycam provide developer APIs or SDKs (verify current terms); Google Cloud, AWS or Azure GPU instances in EU regions can run open-source trainers in batch.
• Pattern: upload video to EU object storage → queue job (GPU container) → write splat to storage → serve with a web viewer; webhook into CMS or listing system.
• Choose B when: you need multi-tenant product scanning, listing-system integration or white-labelled tours.

Path C — Self-host (open source)

• Training: Nerfstudio (with splatfacto) and its gsplat library [3], the original Inria 3DGS reference code (research licence, not for commercial use; check licences), COLMAP/GLOMAP for poses, OpenSplat (Apache/AGPL-style variants, check) and Brush (cross-platform, Rust/WebGPU).
• Viewing/editing: SuperSplat, three.js splat viewers, Spark (three.js renderer), Unity/Unreal/Godot plugins.
• Standards: Khronos has been standardising Gaussian splats in glTF via an extension; adopt it where tools support it to avoid format lock-in [4].
• Hardware: HW2 (one 16-32 GB GPU) trains a room-scale scene in tens of minutes; city blocks and large drone scenes need HW3 or chunked training.

Path D — Custom training

Rarely needed. Worth it for: domain-specific feed-forward models (e.g. interior scenes), anti-artifact priors for reflective or textureless surfaces, or generative completion of unseen regions. Training needs large posed multi-view datasets; academic datasets exist, proprietary capture sets are the differentiator.

State of the art (2026)

• 3DGS and its successors (anti-aliased, 2DGS/surface-aligned variants, compressed formats) are the default for photoreal capture; NeRF variants (Instant-NGP, Zip-NeRF) remain strong in quality benchmarks but lose on rendering speed [1][2].
• Compression (e.g. Niantic's SPZ, PlayCanvas SOG) reduces scene files from hundreds of MB to tens of MB, which makes mobile web delivery workable.
• Large scenes handled via level-of-detail and chunked training; dynamic scenes (4D splats) are research-grade to early-adopter.
• Diffusion priors fill in missing views and clean floaters; generation-to-3D overlaps with GEN-008.
• Limits: reflective, transparent and textureless surfaces; moving people and lighting changes between frames; sparse coverage causes floaters and holes; no true physical material (relighting is limited); limited metric accuracy; mesh extraction loses much of the photoreal quality; standards and tool support still fragment.

Shared resources & combinations

• Shared stack (3d-vision, GPU training, segmentation): GEN-008 (3D asset generation), GEN-009 (architecture visualisation), GEN-018 (virtual staging; staging furniture into splats is an active area), GEN-011 (masking/cleanup), VIS-021 (aerial imagery), ROBO-009 (construction monitoring), ROBO-010 (simulation environments).
• One HW2/HW3 GPU server running a queued trainer can serve several of these; privacy masking models (face and plate blurring) are shared with surveillance-adjacent computer-vision pipelines.
• Downstream: AR try-in-place, VR tours, digital twin platforms, game engines.

Effort, cost & KPIs

• Path A: hours to days; subscriptions per user or per scene credit.
• Path B: 1-3 months for a production pipeline with listing integration; cost driven by GPU minutes per scene and storage/egress.
• Path C: 1-2 months to a working pipeline, plus ops for GPU queue and viewer hosting.
• KPIs: capture success rate (usable scene on first try), processing time per scene, file size and load time on mobile, viewer engagement/dwell time vs. photos, lead or enquiry conversion on listings, scan-to-publish cost, geometric error where measured.

People & change

Who and when. Involve the department lead (sales, marketing, facility or heritage management) and end-users (agents, photographers, product staff) at the start, because capture quality decides the outcome. Bring in the DPO and legal-compliance before scanning occupied homes, offices or venues: residents and staff need notice, consent or a legal basis, and copyright or house rules may restrict scanning. Customers and residents are affected stakeholders and should be asked before capture. Management sets the policy on publication and retention (GUIDE-007).

Works council. Depends. Scanning products or empty properties does not monitor staff. If employees appear in scenes, or scans of workplaces or production halls are used as a digital-twin layer, co-determination under §87(1) no. 6 BetrVG may apply (BAG: objective suitability suffices), and §90(1) no. 3 requires timely information. Agree on masking of staff and no use for performance evaluation. Public sector: staff council.

DPO and DPIA. Sign an AVV (Art. 28 GDPR) with the scan platform and check transfer safeguards for US vendors. A DPIA is likely only for systematic scanning of public or workplace spaces (GUIDE-005).

Literacy, communication, pilot. Support AI literacy (Art. 4 AI Act) with a one-hour capture training (GUIDE-006). Pilot with 5-10 objects and compare enquiry rates against photos (GUIDE-009). Typical concern: misleading listings. Address it with disclosure of edits, unedited capture kept, and human sign-off before publication.

Risks, regulation & compliance

• EU AI Act: typically minimal risk; no Annex III category. Art. 50 transparency applies if the output is presented as depicting a real place but is synthetically altered or completed (deepfake-style manipulation); label edited or generated regions. No Art. 5 issue for property or product capture. Biometric or surveillance use of scans would be a different, sensitive use case and is out of scope here. AI literacy duty (Art. 4) applies.
• GDPR: scenes of homes, offices and public spaces can capture people, faces, number plates, documents on desks and family photos; capturing residents or staff requires a legal basis, notice and masking. Occupied homes: obtain consent from residents and remove personal items. Processing with a US vendor needs transfer safeguards.
• Other law: property and database/copyright (scanning artworks, buildings with copyright, museum objects), Germany's Panoramafreiheit limits, drone rules (EU UAS regulation, EASA), house rules and privacy at hotels and venues, model/licence terms for the scene output.
• Misrepresentation risk: misleading real-estate listings if floaters are hidden or rooms are altered; consumer-protection law applies. Mitigation: disclose edits, keep unedited capture, human sign-off.
• Security: hosting scenes of critical facilities exposes layouts; restrict access and links.

Getting started

1. Pick one concrete scenario (e.g. five properties or ten products) and define the delivery format (web embed, AR, BIM review).
2. Capture the same subject with Scaniverse (free, on-device) and Polycam or Luma; compare quality, export formats and terms.
3. Write a capture guide (slow movement, overlap, even lighting, no moving people) and a privacy checklist (masking, consents).
4. Prototype a self-hosted pipeline with Nerfstudio/gsplat plus SuperSplat on one GPU to check cost and lock-in.
5. Publish pilots, track engagement and cost per scene, then decide between A and B/C.

Sources

1. Kerbl et al. — 3D Gaussian Splatting for Real-Time Radiance Field Rendering (SIGGRAPH 2023) — https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
2. Mildenhall et al. — NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis — https://www.matthewtancik.com/nerf
3. Nerfstudio and gsplat documentation — https://docs.nerf.studio/
4. Khronos Group — glTF specification and extensions repository — https://github.com/KhronosGroup/glTF
5. Scaniverse (Niantic Spatial) — https://scaniverse.com/
6. PlayCanvas SuperSplat — https://github.com/playcanvas/supersplat
7. COLMAP — https://colmap.github.io/
8. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
9. Regulation (EU) 2016/679 (GDPR) — https://eur-lex.europa.eu/eli/reg/2016/679/oj
10. Zillow — press release introducing SkyTour (15 Jul 2025) — https://zillow.mediaroom.com/2025-07-15-Summer-just-got-hotter-Zillow-debuts-five-powerful-new-features