Animation Synthesis from Story Scripts

Sep 1, 2021
system overview
Figure 1. Synthesis Pipeline

As shown in Figure 1, we develop an automated production tool that can efficiently generate animations from natural language-described story scripts. The synthesis pipeline contains two stages:

  • Utilizing our animation model, which is a complex system that integrates NLP analysis and a Unity-based graphics engine, to generate an animation clip from a story script.
  • Making further adjustments to the animation using an interactive editing tool, such as CapCut for TikTok.

Below are several animation demos created by our system from story scripts written in Chinese:

Video 1. Animation generated from a story script about Naming.
Video 2. Animation generated from a story script about Rent Collection.
Video 3. Animation generated from a story script about the Chairman's Inspection.
Video 4. Animation generated from a story script about Peppa Pig.
Video 5. Animation generated from a description of a scene about a Castle Dinner.
Video 6. Animation generated from a story script about Ultraman.

Besides, we generated more animation videos using an earlier version of our product and shared them on Douyin (ID: heidongdonghua) – the Chinese TikTok, on an irregular basis.

Tenglong Ao 敖腾隆
Tenglong Ao 敖腾隆
Ph.D Candidate

Publications

Animation Synthesis from Story Scripts
We develop a language-assisted animation creation product that enables users (e.g., Tiktokers and kids) to effortlessly create simple animations by specifying natural language-described story scripts.
Nice Colleagues, Tenglong Ao
Project
Animation Synthesis from Story Scripts