Code-driven video production workshop

The starting point

A workshop built to produce short vertical videos without going back through an editing suite: the style is written once, then reused from one video to the next.

The problem

A polished vertical video — word-by-word captions, number cards, voice-over — takes hours of manual editing, and conventional editing software makes it hard to reuse the same style. AI-generated shots add problems of their own: unreadable text, watermarks, incompatible colour spaces.

The answer

The workshop treats video as code: compositions written in React and driven by data lay an animated overlay on top of the footage, without touching the edit or the music. Scripts automate the upstream steps — generating shots, synthetic voice-over, transcription — and every step is documented so it can be replayed identically.

• Word-by-word animated captions, generated from an automatic transcript
• Reusable overlays: caption bar, number cards, badges, markers
• Vertical 1080 × 1920 composition driven by a single data file
• AI-generated shots and synthetic voice-over built into the chain
• Deterministic rendering, then automatic checks on picture and sound

Under the hood

Every video follows the same flow: prepare the footage, transcribe it word by word, then turn that transcript into data (words, cards, sound effects). The Remotion compositions, written in React and rendered frame by frame, read this data and lay the overlay on top. Upstream, scripts drive the image, video and voice generation models; downstream, ffmpeg assembles, mixes and converts colour spaces. Everything is indexed on the frame number, which makes renders reproducible.

The notable challenge

Converting HDR footage shot on a phone down to SDR, automatic peak detection made the contrast drift over the course of a shot — a flaw invisible on any single frame. The problem was found by comparing several frames spread across the duration, then fixed by freezing the conversion curve.

My role

Odilon designed the production chain and built it with Claude Code: compositions and overlays, automation scripts, documentation of each step, art direction and render checks.

Where the project stands

The captioning and overlay chain works and is used to produce videos end to end. Current work is on replacing filmed elements with generated shots.

• Finish replacing filmed elements with generated shots
• Automate publication scheduling

The project’s tools

Video rendering

Remotion 4, React, TypeScript

Overlays

HyperFrames, d3-geo, topojson

AI generation

Veo 3.1 (Google Flow), Seedance, Nano Banana

Synthetic voice

ElevenLabs, edge-tts

Transcription

Whisper

Processing

ffmpeg and ffprobe, Python, Node.js

Orchestration

Claude Code (specialised subagents)