Screen production is a chain of expensive visual decisions: look development, boards, previs, shooting, VFX. Generative image and video models now compress the early stages from days to hours and, in selected shots, deliver final pixels that go through the normal compositing pipeline. Benefits are faster iteration and cheaper shots that were previously out of reach; the price is tighter rights, consent and labour-agreement management, plus a controllability gap that still needs VFX artists.
Problem & value
- Today: concept artists and storyboard artists draw boards by hand, previs teams build rough 3D animatics, and VFX houses spend weeks on set extensions, destruction or crowds. Directors and producers decide on budget and look late, when change is costly. Commercials (ads) have the same chain with shorter schedules.
- AI value: many look and framing alternatives early (pitch decks, moodboards, boards), script-to-previs without full 3D builds, and for some shot types (background set extension, debris, atmospherics, crowds, de-ageing) final-quality output at lower cost and time.
- Evidence: Netflix publicly described its use of generative AI for final footage in the Argentine series El Eternauta (2025), a building-collapse sequence produced with its in-house Eyeline Studios, and published guidance on generative-AI use for productions [1][2]. Lionsgate announced a custom-model partnership with Runway in 2024 [3]. Vendor-reported time and cost savings exist but are not independently benchmarked; measure your own.
Variants
- Development and pitch: moodboards, key art, lookbooks, animated pitch trailers.
- Storyboards and animatics: script-to-board with character and location consistency, camera notation.
- Previs and techvis: generated clips as motion references, depth/camera-controlled generation, tests before expensive shoots.
- Final-pixel generative VFX: set extension, destruction/simulation-like effects, crowds, matte paintings, de-ageing, video-to-video restyling of plates.
- Advertising: agencies produce spots with generated backgrounds, product environments and versioned cut-downs (see GEN-002 for short social video).
How it works
- script / brief
- LLM breakdown (scenes, shots, characters, locations, continuity bible)
- concept art & style frames (image-gen, style/character references, LoRA)
- storyboard & animatic (consistent characters, shot lists, camera notation)
- previs clips (image-to-video, camera control, depth/pose conditioning)
- hero shots: video-gen / video-to-video on real plates + masks, depth, AOVs
- VFX pipeline: upscaling, denoise, roto, relight, compositing (Nuke, Flame, Unreal)
- QC, legal clearance, provenance (C2PA), disclosure
- editorial / online / delivery
- Key components: text/image-to-video and video-to-video models; camera and motion control; reference-based consistency (characters, props, locations); inpainting/outpainting; upscaling; depth and segmentation for compositing; LLMs for breakdowns and prompt libraries.
- Human-in-the-loop: director/DoP approve the look; VFX supervisor decides which shots may be generative; artists curate, fix and composite; legal/clearance checks training-data indemnity, likeness and music; union and consent obligations are checked per performer. Generated frames are plates, not final delivery, until QC.
Data preparation
The material is scripts, shot lists, reference stills, plates, 3D and depth data, actor scans and voice recordings. It lives in production asset management (e.g. ShotGrid), NAS and file shares, mail threads, cloud drives and often on individual artists' machines. Contracts and clearances sit in the DMS or on paper.
Typical gaps: unknown rights status of reference images, missing consent records for actor scans and voices, scattered naming without shot or version IDs, mixed resolutions and colour spaces, and no separation of licensed, owned and AI-generated assets.
Preparation steps: (1) a rights register per asset (owned, licensed, AI-generated, consent scope); (2) consistent shot/version naming and colour pipeline (ACES/EXR); (3) curate a style bible and approved reference set per project for LoRA or image-guided generation; (4) digitise and link contracts and consent forms to the assets; (5) pseudonymise or exclude personal data not needed; (6) set up provenance logging. Tools: ShotGrid or similar, ExifTool, C2PA tooling.
Rough share of effort: about 25 to 35 percent. See PREP-010 for tagging, PREP-016 for gold-standard examples, PREP-024 for data classification rules and PREP-025 for legal-basis checks.
Implementation paths
Which path for whom
- smb: Path A, because small studios and agencies get commercially licensed image and video tools (e.g. Firefly, Runway) at subscription prices and mostly need standard concept and storyboard work.
- enterprise: Path B, because studios need cloud model APIs inside their asset management, compositing and rights-tracking pipeline, with contractual control over training use.
- developer: Path C, because open video and image models with ComfyUI allow custom pipelines, LoRA training on owned material and no upload of unreleased footage.
Path A: Buy (off-the-shelf)
- Generative video/creative suites: Runway (Gen-4 family, Aleph video editing), Luma (Ray family, incl. HDR output), Google Veo (via Flow/Vertex AI), OpenAI Sora (discontinued in 2026), Kling, Adobe Firefly video (commercially-safe positioning), Moonvalley Marey (trained on licensed data), Stability AI tools.
- EU-relevant: Black Forest Labs FLUX (Germany) for stills, Mistral for text work, Stability (UK), Photoroom (France), Wonder-style local studios; EU hosting and data-processing terms vary, so check contract terms.
- Choose A when: concept, boards and previs for pitch or planning, or small ad production without a studio pipeline. Insist on indemnification, no-training-on-your-data terms and exportable high-bit-depth formats.
Path B: Build on cloud APIs (recommended for enterprise)
- Models via API: Google Veo (Vertex AI, EU regions), Runway API, Luma API, Kling API, FLUX for still images, Azure/Google/AWS regional hosting where available.
- Orchestration: ComfyUI-style graph workflows or custom services calling APIs; asset management (ShotGrid/Flow Production Tracking, ftrack), versioning, and automatic metadata capture (prompt, seed, model version, source assets).
- Integration: deliver EXR/ProRes into Nuke or Unreal; use depth/mask passes from the plate for compositing; add C2PA content credentials.
- Choose B when: you need pipeline integration, versioned provenance and per-project cost control.
Path C: Self-host (open source)
- Video models: Wan 2.x (Alibaba, open weights), HunyuanVideo (Tencent), LTX-Video (Lightricks), Mochi; image models: FLUX.1 (open-weight variants, check licence), Stable Diffusion 3.x/SDXL; control via ControlNet, depth/pose conditioning, LoRA/IP-Adapter for consistency.
- Stack: ComfyUI, diffusers, upscalers (Real-ESRGAN family), SAM-class segmentation for masks, Blender/Unreal for hybrid previs.
- Hardware: HW3 for single-artist workstations (image and short clips with offloading), HW4 for render-farm style throughput and fine-tuning. Check licences for commercial use of each model.
- Choose C when: unreleased IP must not leave the premises (strict NDA, studio security requirements) or for custom fine-tuned looks.
Path D: Custom training
Studios fine-tune on their own IP: character/style LoRAs, a house-style model trained on owned footage or art (cf. Lionsgate-Runway). Needs rights to every training asset, curated data of a few hundred to thousands of images or clips, and legal sign-off. Performer likeness models only with explicit contractual consent and scope limits.
State of the art (2026)
- Quality: leading video models deliver 1080p+ clips of a few seconds to about a minute with strong motion, audio-native variants and improving physics; HDR/EXR-class output and camera/depth control are arriving in professional tiers.
- Consistency: character and set consistency across shots needs reference images, LoRAs or multi-shot features; it remains the main manual effort.
- Limits and failure modes: short clip length, fine control of acting performance, text/logos, hands and complex interaction physics, temporal flicker, resolution and bit depth (needs upscaling and grading), non-determinism (reproducibility problems for reshoots), unclear copyright status of outputs, and hidden training-data risk.
- Where it works best (2026): background/environment shots, effects elements, previs and boards. Hero close-ups with dialogue are still mostly shot or CG.
Shared resources & combinations
- Shared stack: video-gen and image-gen with GEN-002 (marketing video), GEN-011 (retouching/upscaling), GEN-012 (editing), GEN-008 (3D assets for Unreal previs), GEN-022 (motion capture from video).
- Voice/dubbing: SPEECH-007 for localisation; GEN-007 for score and sound design; GEN-003 where presenters are avatars.
- Trust and security: SEC-011 (deepfake detection) and provenance (C2PA) are the control counterpart.
Effort, cost & KPIs
- Path A: days to weeks; subscription or credit based; credit consumption rises quickly with iterations.
- Path B: 2-4 months for a pipeline with asset tracking, provenance and compositing handoff.
- KPIs: time from script to approved boards/previs, iterations per approved shot, share of generated shots passing QC first time, artist hours saved against baseline VFX bid, cost per second of final footage, incident count (rights/consent complaints), audience-disclosure compliance.
People & change
Involve management (strategy, risk, budget) and the production or creative lead early, then the artists, directors and editors who work with the tools daily. Legal or compliance and HR are essential for actor and crew contracts, consent for scans and voices, collective agreements and union rules. IT and suppliers (VFX vendors, rights holders) join once the pipeline is defined. The data protection officer must be involved because biometric-like scans and voices are personal data; an AVV is required for each cloud service and a DPIA is likely for large-scale face or voice processing.
Works council: depends. Creative AI tools are not designed to monitor staff, but platform logging can make them suitable for monitoring (§87(1) no. 6 BetrVG, BAG case law), and changed work processes must be reported in good time (§90(1) no. 3). Freelancers are outside the council, so also agree rules in contracts.
AI literacy: offer hands-on workshops that show what the models can and cannot do (Art. 4 AI Act: measures that support literacy), plus labelling duties under Art. 50. Pick one pilot production or one department such as storyboards. Typical concern: "AI replaces artists". Address it openly: define which stages use AI, keep artists as decision makers and credit their work. Humans approve every final shot.
See GUIDE-008 for acceptance, GUIDE-004 for the works council and GUIDE-014 for AI Act duties.
Risks, regulation & compliance
- EU AI Act: typically limited risk; Art. 50 transparency applies to deepfakes and synthetic image/video content: providers must machine-readable mark outputs, deployers must disclose deep fakes, with a lighter regime for evidently artistic, creative or fictional works (disclosure must not hamper the work). The Commission is preparing a code of practice on marking and labelling [4][5]. General-purpose-model providers have copyright and training-data transparency duties (Art. 53) that affect vendor due diligence.
- Art. 5 flags: do not use generation of manipulative content that exploits vulnerabilities or subliminal techniques, and no biometric categorisation or emotion inference on cast or crew; none is needed here.
- GDPR: actor faces, voices and body scans are personal data (biometric when used for unique identification); need a legal basis, explicit contractual consent, purpose limitation, retention/deletion, DPIA for large-scale scanning, and processor contracts with EU transfer safeguards.
- Copyright and rights: unresolved questions on training data, EU TDM opt-outs (Art. 4 DSM Directive) and protectability of AI-assisted works; prefer models with licensed-data or indemnified terms; keep human authorship documented.
- Labour agreements and personality rights: SAG-AFTRA, WGA and national agreements (and national laws such as Germany's personality rights) restrict digital replicas and AI use; obtain consent and compensation terms per use, and no replica reuse beyond the scope.
- Mitigations: AI-use register per project, vendor indemnity checks, C2PA credentials, review gates, ban on training vendors with your IP, legal review before delivery.
Getting started
- Define permitted uses with legal and unions/guilds: concept-only, previs, background VFX; list excluded uses (performer likeness).
- Run a two-week pilot on previs or boards for one project; track iterations and time saved.
- Test two or three tools on one real VFX shot type (e.g. set extension) and send results through your normal Nuke/Unreal QC.
- Add asset tracking, prompt/seed logging and C2PA provenance before using output in delivery.
- Draft the vendor checklist: data use, indemnity, EU hosting, output format, and disclosure rules (AI Act Art. 50).
Sources
- Netflix Tudum - newsroom (El Eternauta and generative AI, 2025) — https://www.netflix.com/tudum
- Netflix Partner Help Center — Using Generative AI in Content Production — https://partnerhelp.netflixstudios.com/
- Runway — Lionsgate partnership announcement — https://runwayml.com/news/runway-partners-with-lionsgate
- Regulation (EU) 2024/1689 (AI Act), Art. 50 — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- European Commission — AI Act regulatory framework and codes of practice — https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
- Coalition for Content Provenance and Authenticity (C2PA) — https://c2pa.org/
- Wan open video models — https://github.com/Wan-Video/Wan2.1
- ComfyUI — https://github.com/comfyanonymous/ComfyUI
- Runway documentation — https://docs.dev.runwayml.com/