StoryMem: Generating Coherent Multi-Shot Long Videos with Memory in 2025

As we close out 2025, AI video generation has made remarkable strides. Tools that once struggled with short, inconsistent clips can now produce minute-long narratives with cinematic flair. One standout advancement is StoryMem, a framework that enables multi-shot long video storytelling while maintaining impressive character consistency and visual quality.

Released just days ago in late December 2025, StoryMem builds on powerful single-shot video diffusion models to create coherent stories. If you’re exploring AI for filmmaking, content creation, or research, this guide dives deep into how it works, why it matters, and how to get started.

What Makes StoryMem Different?

Most video generation models excel at single shots—short clips of 5-10 seconds. Extending them to full stories often leads to drifting characters, changing styles, or abrupt transitions.

StoryMem takes inspiration from human memory. It treats long-form storytelling as an iterative process: generate one shot at a time, while referencing “memories” from previous shots.

This Memory-to-Video (M2V) approach keeps a compact bank of keyframes from earlier generations. These keyframes condition the next shot, ensuring characters look the same, scenes evolve naturally, and the overall narrative flows smoothly.

The result? Minute-long videos (typically 40-60 seconds, with 8-12 shots) that feel like directed short films, complete with consistent protagonists and professional-looking cinematography.

Project demos showcase everything from realistic urban tales to fantasy adventures, highlighting cross-shot coherence.

How StoryMem Works: A Step-by-Step Breakdown

StoryMem operates shot-by-shot, much like a filmmaker shooting scenes sequentially.

The Memory Bank

  • Stores selected keyframes from prior shots.
  • Limited size (default: 10 shots) for efficiency.
  • Dynamically updated to retain the most relevant visual information.

Memory-to-Video Generation

Built on the Wan2.2 foundation models (text-to-video and image-to-video variants):

  1. First shot: Generated using standard text-to-video (T2V).
  2. Subsequent shots: Memory keyframes are encoded, concatenated to noisy latents, and injected with negative RoPE shifts for positional awareness.
  3. Lightweight LoRA fine-tuning adapts the base model to understand memory conditioning.

This keeps the high fidelity of pretrained models without expensive full retraining.

Smart Keyframe Selection

After each shot:

  • Semantic matching using CLIP features to pick frames most relevant to the story.
  • Aesthetic filtering with HPSv3 to avoid low-quality or unstable frames.

This ensures the memory bank stays informative and reliable over long sequences.

Extensions for Better Control

  • MI2V: Adds first-frame image conditioning for seamless transitions when no scene cut.
  • MM2V: Uses first 5 motion frames for even smoother dynamics.
  • MR2V: Starts with user-provided reference images as initial memory—perfect for custom characters.

These make StoryMem versatile beyond pure text prompts.

Recent demos include reference-guided stories like custom portraits in narrative settings.

Hands-On: Installing and Running StoryMem

The code is open-source and straightforward to set up. Here’s a complete guide.

Setup Steps

  1. Clone the repository:

    git clone --single-branch --branch main git@github.com:Kevin-thu/StoryMem.git
cd StoryMem

  2. Create environment:

    conda create -n storymem python=3.11
conda activate storymem
pip install -r requirements.txt
pip install flash_attn

Downloading Models

Model Hugging Face Link Purpose
Wan2.2 T2V-A14B Wan-AI/Wan2.2-T2V-A14B Base text-to-video
Wan2.2 I2V-A14B Wan-AI/Wan2.2-I2V-A14B Base image-to-video
StoryMem LoRA Kevin-thu/StoryMem Memory conditioning weights

Use the CLI for easy download:

pip install "huggingface_hub[cli]"
huggingface-cli download Wan-AI/Wan2.2-T2V-A14B --local-dir ./models/Wan2.2-T2V-A14B
huggingface-cli download Wan-AI/Wan2.2-I2V-A14B --local-dir ./models/Wan2.2-I2V-A14B
huggingface-cli download Kevin-thu/StoryMem --local-dir ./models/StoryMem

Two LoRA options:

  • Wan2.2-MI2V-A14B: M2V + image conditioning
  • Wan2.2-MM2V-A14B: M2V + motion frame conditioning

Generating Your First Video

Run the example:

bash run_example.sh

It automatically:

  • Generates the first shot with T2V.
  • Proceeds shot-by-shot with memory updates.
  • Saves individual clips and a concatenated final video.

Key parameters to tweak:

Parameter Description Default
story_script_path Path to JSON script Example file
output_dir Save location ./results
size Resolution (e.g., 832×480) 832*480
max_memory_size Max shots in memory 10
mi2v / mm2v Enable smooth transition modes False
seed For reproducibility 0

Recent minor updates (as of December 26, 2025) fixed safetensors loading for better compatibility.

Crafting Effective Story Scripts

StoryMem uses structured JSON scripts. The new ST-Bench provides 30 diverse examples (300 prompts total), covering realistic, fairy-tale, and cultural styles.

Script Structure

  • story_overview: Brief summary.
  • scenes: Array with scene_num, video_prompts (list of strings), cut (True/False array).

Each prompt (1-4 sentences) should cover:

  • Characters: Appearance, age, attire.
  • Actions: Gestures, expressions, interactions.
  • Scene: Location, lighting, props.
  • Atmosphere: Mood, colors.
  • Camera: Shot type (wide, close-up), simple movements.

For non-cut shots (cut=False), ensure seamless continuation from the previous frame.

Tip: Explicitly describe characters in every prompt to help memory matching, especially in multi-character stories.

Example snippet:

{
  "story_overview": "A day in the life of a street musician...",
  "scenes": [
    {
      "scene_num": 1,
      "video_prompts": ["Morning in apartment...", "Looking out window..."],
      "cut": [true, false]
    }
    // more scenes
  ]
}

Performance and Limitations

StoryMem shines in cross-shot consistency while preserving Wan2.2’s visual quality. It outperforms decoupled keyframe methods and avoids the heavy costs of joint long-video training.

Current limitations:

  • Multi-character scenes can occasionally confuse memory retrieval—mitigate with detailed prompts.
  • Motion speed differences may cause minor transition artifacts; future overlapping frames could help.

As of late 2025, community integrations (e.g., ComfyUI nodes) are emerging, expanding accessibility.

Frequently Asked Questions

How long can videos be?

8-12 shots at 5 seconds each yield 40-60 seconds. Longer is possible by increasing memory size.

What hardware do I need?

At least 24GB VRAM recommended (e.g., RTX 4090 or A100) for smooth inference.

Can I use custom characters?

Yes—via MR2V. Place reference keyframes in the output directory.

Is it free to use?

Open-source with cc-by-nc-4.0 license; base Wan2.2 is also openly available.

How does it compare to commercial tools?

Focuses on open-source coherence for storytelling, complementing closed models.

Citation

For research:

@article{zhang2025storymem,
  title={StoryMem: Multi-shot Long Video Storytelling with Memory},
  author={Zhang, Kaiwen and Jiang, Liming and Wang, Angtian and Fang, Jacob Zhiyuan and Zhi, Tiancheng and Yan, Qing and Kang, Hao and Lu, Xin and Pan, Xingang},
  journal={arXiv preprint arXiv:2512.19539},
  year={2025}
}

StoryMem represents a thoughtful step forward in AI-driven storytelling. By mimicking selective human memory, it bridges short clips into meaningful narratives—all while staying efficient and open.

Whether you’re a creator experimenting with AI films or a researcher pushing video generation boundaries, StoryMem offers practical tools and inspiring results. Give it a try and see how memory transforms video AI.

(Word count: approximately 3,450)