Making LLMs watch video instead of just reading the transcript

What it does

claude-real-video is a local Python preprocessor that turns videos into LLM-readable artifacts. It ingests a URL or local file, detects actual scene changes instead of sampling blindly, strips near-duplicate frames with a sliding-window pixel comparison, and pairs the results with a transcript. Everything happens on your machine; you decide what gets pasted into Claude, ChatGPT, or Gemini afterwards.

The interesting bit

The deduplication is intentionally primitive in a clever way: it compares downscaled RGB pixels rather than perceptual hashes, which the author notes can miss flat-color shifts and equal-luma hue changes. A rolling window also prevents A-B-A cutaways from re-sending shots the model has already seen, keeping token counts low without missing actual visual changes.

Key highlights

• Scene-change detection plus a density floor catches both fast cuts and slow pans, avoiding the fixed-interval trap.
• Transcript extraction prefers existing subtitle tracks over re-transcribing with Whisper when available.
• Optional --grid output packs keyframes into contact sheets so models read sequential context instead of scattered stills.
• Runs fully offline after dependencies are present; only the frames and text you choose get sent to an LLM.
• Doubles as a general-purpose keyframe extractor even if you never touch an LLM.

Caveats

• Requires ffmpeg and ffprobe installed separately; they are not bundled or pip-installable.
• Re-running the tool overwrites the output directory without warning.
• Optional audio transcription needs an extra [whisper] dependency layer.

Verdict

Grab it if you want a local, token-efficient way to make video legible to any LLM. Pass if you need the tool to do the actual analysis for you—this only prepares the context, it does not chat with the model.