Short films from a single sentence

What it does

StoryTeller is a workflow engine that wires three existing generative models into a single pipeline. Feed it an opening sentence and it automatically writes the rest of the plot, illustrates each sentence with Stable Diffusion, and narrates the script with neural TTS. The output is a short composite video with audio, subtitles, and the intermediate still frames.

The interesting bit

The project skips end-to-end training and instead treats story generation as an assembly line. The clever part is the sentence-level granularity: each line of generated text becomes its own image prompt, so the story effectively storyboards itself without a human in the loop.

Key highlights

• Chains gpt2, stable-diffusion-2, and Coqui TTS into one automated pipeline
• Produces one illustration per sentence, then syncs narration and frames into video
• CLI and Python API let you mix model precisions and split writer/painter across separate GPUs
• Defaults are geared toward instant demos, but all three model slots are swappable

Caveats

• Apple Silicon support is explicitly work-in-progress: PyTorch stable 2.0.1 crashes on torch.cumsum under MPS, so nightly builds are currently required
• macOS users face a dependency snag because mecab-python3 lacks Apple Silicon wheels and needs a manual Homebrew workaround

Verdict

Useful for rapid multimodal prototyping or automated content experiments. Skip it if you need fine-grained narrative or visual control—the pipeline trades authorship for convenience.