yukkuri-video-generator: The Repo That Builds Videos by Listening First

A Python pipeline for Yukkuri commentary that turns script text into rendered MP4s only after the voice exists, the duration is known, and the timeline can be locked to reality.

6 to 8 min read • View on GitHub • More from neo-a11y

A wide black-ink editorial scene shows a script entering a machine on the left, then passing through a voice synthesis chamber, a waveform measuring gauge, and finally a finished video reel on the right. The waveform acts like a ruler that sets subtitle spacing and character timing, explaining the repo’s audio-first composition model.
The whole pipeline listens before it renders. Audio is not an output detail here. It is the clock that sets everything else.
Key Takeaways

Why this repo feels more like a production system than a script

Most lightweight video tools guess timing from text and then force the visuals to fit. This repo does the opposite. It synthesizes the voice first, reads the actual WAV duration, and only then builds the rest of the timeline around something real.

That order matters. It turns lip-sync, subtitles, and sprite placement from estimates into dependencies, which is a much safer way to automate a media workflow.

The diagram makes the central idea visible. Audio is not one more asset in the pipeline. It is the timing source that every later step depends on.

Yukkuri videos, without the manual assembly line

If you do not already know the format, Yukkuri videos are built from static character avatars, synthetic voices, and subtitles. The hard part is not any single element. It is stitching them together repeatedly without spending your life in a timeline editor.

WorkflowInput styleMain job solvedWhere timing comes fromManual editingBest use caseMain limitation
Manual YMM4 assemblyScript plus assetsHand-built Yukkuri episodesHuman timing and editor judgmentHighFine-grained creator controlSlow and repetitive
This repoScript in, MP4 outAutomated Yukkuri renderingMeasured audio durationLowBatch production from text scriptsDepends on VOICEVOX and local assets
AutoYukkuriLive voice commentaryConvert spoken footage into YMM4 projectsSpeech recognition timingMediumCreators who start from recorded narrationStill centered on YMM4 workflows
KuchiPakuTiming label filesAutomate lip-sync and expression placementExternal label filesMediumTimeline assistance inside YMM4Solves a narrower problem
Bespoke automation scriptsCustom pipelineEnd-to-end content generationVaries by stackLow to mediumTeams with unique publish workflowsUsually fragile and project-specific

How the pipeline actually moves

The codebase is cleanly separated. The parser turns raw text into dialogue objects. The voice layer calls VOICEVOX. The subtitle layer wraps text for the screen. The composer stitches everything into a single render pass.

# The important pattern is dependency order, not a specific framework.
parsed = script_parser.parse(script_text)
for line in parsed:
    audio_file = voice_synthesizer.synthesize(line.text, speaker_id=line.speaker_id)
    duration = AudioFileClip(audio_file).duration
    subtitle = subtitle_generator.build(line.text, duration=duration)
    video_composer.add_segment(audio=audio_file, subtitle=subtitle, duration=duration)

final_clip = video_composer.compose()
A close-up black-ink diagram shows a real waveform being measured like a precision instrument. Markers snap into place after the audio length is read, and subtitle blocks and character sprites align to those markers on the right. The scene explains why measured audio duration makes the timeline deterministic instead of guessed.
The timing trick is small, but it changes everything. The render is built around the waveform that actually came out of synthesis, not around an estimate.

Why the config file matters more than it looks

The YAML layer is not just convenience. It is the reason the project can stay extensible. Character positions, speaker IDs, and layout settings live outside the core logic, so new setups do not require rewriting the engine.

video:
  resolution:
    width: 1920
    height: 1080
characters:
  reimu:
    speaker_id: 1
    position:
      x: 180
      y: 620
  marisa:
    speaker_id: 2
    position:
      x: 1460
      y: 620
voicevox:
  host: "http://127.0.0.1:50021"

The repo is also documenting how AI should collaborate with code

I created an automated tool that takes a script and outputs the main video, short video, and thumbnail in under 7 minutes. Since I'm also instructing ChatGPT to create the script, I feel like I've become a machine that just passes scripts to the program and hits a single command.

This repository’s meta files matter because they treat process as part of the product. The code is not only describing how to render a video. It is also telling collaborators, including AI collaborators, how to work inside the repo.

That is a bigger shift than it first appears. A project with explicit maintenance rules, implementation notes, and workflow constraints is easier to extend safely than a pile of scripts with no operating model.

What this competes with, and what it avoids

The comparison is not about which tool is better in the abstract. It is about where each one sits in the workflow. Some tools help you start from a live recording. Some help you lip-sync inside YMM4. This repo tries to remove the editing middle layer entirely.

ToolPrimary inputOutputTiming modelEditing surfaceWhat it avoids
Manual YMM4Script and assetsHand-authored projectHuman decidedFull editor UIAutomation
This repoScript textRendered MP4Measured audio durationConfiguration files and Python modulesGuessing timing
AutoYukkuriSpoken narrationYMM4 projectWhisper-derived alignmentYMM4-compatible workflowManual transcription
KuchiPakuLabel files and TTS timingLip-sync automation inside YMM4Label drivenExpression timeline toolingRebuilding the whole video stack
Bespoke scriptsVariesVariesVariesUsually code onlyProject-specific repeat work

That niche is the whole story. It is not trying to replace every Yukkuri tool. It is trying to make one narrow path much faster and much more reliable.

Where it stands today

The repository reads like an early prototype with strong intent. The structure is clear, the separation of concerns is sensible, and the timing model is genuinely smart. The missing pieces, like tests and complete asset coverage, make it feel unfinished in the honest way prototypes should.