Construction of an animated multi-shot audio-visual dataset for educational content generation

doi:10.11959/j.issn.2096-0271.BDR26166

Chinese

您当前的位置：

首页 >

文章列表页 >

Construction of an animated multi-shot audio-visual dataset for educational content generation

更新时间：2026-09-20

- Construction of an animated multi-shot audio-visual dataset for educational content generation
- Big Data Research (2026)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.BDR26166
  CLC： TP391.41
- Received：30 April 2026，
  
  Revised：2026-09-15，
  
  Accepted：20 September 2026，
- 稿件说明：
移动端阅览
Construction of an animated multi-shot audio-visual dataset for educational content generation[J/OL]. Big Data Research, 2026.
DOI：

Construction of an animated multi-shot audio-visual dataset for educational content generation[J/OL]. Big Data Research, 2026. DOI： 10.11959/j.issn.2096-0271.BDR26166.

摘要

面向情景化教育视频生成，现有音视频数据集在多镜头结构、动画域适配和音画一致性方面仍存在不足。构建动画多镜头音视频数据集 Edu-Anime-MultiShot，并形成“场景—镜头”层次组织、分阶段质量过滤和“全局—局部”结构化标注相结合的半自动构建方法。方法重点设计动画多镜头数据治理规则与双阈值门控，并通过场景级角色原型和统一角色 ID 约束降低跨镜头指代漂移。与 TalkCuts 相比，该数据集规模较小，但增加双层结构、对白—角色映射和角色音色信息。人工抽样显示，画外音比例由 24.6% 降至 1.2% 以内。结果表明，该流程可改善数据结构化程度和音画一致性，为教育内容生成提供可复用的数据治理方案。

Abstract

Contextualized educational video generation requires training data with multi-shot structure

animation-domain adaptation

and reliable audio-visual correspondence. To address these requirements

Edu-Anime-MultiShot was constructed

and a semi-automatic construction method integrating scene-shot hierarchical organization

staged quality filtering

and global-local structured annotation was developed. Instead of modifying backbone models

task-specific governance rules and threshold design were emphasized. Visual-subject coverage within each speech interval was used as a local constraint

and the proportion of qualified speech duration was used as a global constraint; a two-level gate was adopted to balance sample retention and audio-visual consistency. Scene-level character prototypes and shot-level unified character IDs were used to reduce cross-shot reference drift. Compared with TalkCuts

which contains more than 500 h of multi-shot human speech videos

the constructed dataset is smaller but provides a scene-shot hierarchy

dialogue-character mapping

and character-level voice attributes for animated educational scenarios. Manual sampling showed that off-screen speech decreased from 24.6% in raw candidates to below 1.2% after governance. The resulting dataset provides a reusable data-governance and structured-annotation framework for educational animation generation

while the platform results should be interpreted as observational evidence rather than causal proof of learning improvement.

关键词

Keywords

references

Wang S , Xu T L , Li H , et al . Large language models for education: A survey and outlook [J ] . arXiv preprint arXiv: 2403.18105 , 2024 .

Kasneci E , Sessler K , Küchemann S , et al . ChatGPT for good? On opportunities and challenges of large language models for education [J ] . Learning and Individual Differences , 2023 , 103 : 102274 .

Vajjala S , et al . Exploring the role of generative AI in second language education: Insights for instruction, learning, and assessment [J ] . Language Learning & Technology , 2024 , 28 ( 1 ): 1 - 15 .

Xue H , Hang T , Zeng Y , et al . Advancing high-resolution video-language representation with large-scale video transcriptions [C ] // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2022 : 5036 - 5045 .

Wang Y , He Y , Li Y , et al . Internvid: A large-scale video-text dataset for multimodal understanding and generation [J ] . arXiv preprint arXiv: 2307.06942 , 2023 .

Chen T S , Siarohin A , Menapace W , et al . Panda-70m: Captioning 70m videos with multiple cross-modality teachers [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2024 : 13320 - 13331 .

Chen Z , Liu H , Yu W , et al . M3AV: A multimodal, multigenre, and multipurpose audio-visual academic lecture dataset [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 2024 : 9041 - 9060 .

Rossetto L , Bailer W , Dang-Nguyen D T , et al . The castle 2024 dataset: Advancing the art of multimodal understanding [C ] // Proceedings of the 33rd ACM International Conference on Multimedia . 2025 : 12629 - 12635 .

Chen J , Wang Z , Zeng A , et al . Talkcuts: A large-scale dataset for multi-shot human speech video generation [J ] . arXiv preprint arXiv: 2510.07249 , 2025 .

Soucek T , Lokoc J . Transnet v2: An effective deep network architecture for fast shot transition detection [C ] // Proceedings of the 32nd ACM International Conference on Multimedia . 2024 : 11218 - 11221 .

Girdhar R , El-Nouby A , Liu Z , et al . Imagebind: One embedding space to bind them all [C ] // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 2023 : 15180 - 15190 .

Lin B , Zhu B , Ye Y , et al . Video-LLaVA: Learning united visual representation by alignment before projection [J ] . arXiv preprint arXiv: 2311.10122 , 2023 .

Yang Z , Yu Y , Zhao Y , et al . Timeexpert: An expert-guided video llm for video temporal grounding [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision . 2025 : 24286 - 24296 .

Wang Q , Shi X , Li B , et al . Multishotmaster: A controllable multi-shot video generation framework [J ] . arXiv preprint arXiv: 2512.03041 , 2025 .

Radford A , Kim J W , Xu T , et al . Robust speech recognition via large-scale weak supervision [C ] // International conference on machine learning . PMLR , 2023 : 28492 - 28518 .

Redmon J , Divvala S , Girshick R , et al . You only look once: Unified, real-time object detection [C ] // Proceedings of the IEEE conference on computer vision and pattern recognition . 2016 : 779 - 788 .

曾志平 , 张旭龙 , 瞿晓阳 , 等 . 深度伪造音频生成与鉴伪技术综述 [J ] . 大数据 , 2025 , 11 ( 5 ): 130 - 151 .

Zeng Z , Zhang X , Qu X , et al . Review of deepfake audio generation and detection technologies [J ] . Big Data Research , 2025 , 11 ( 5 ): 130 - 151 .

Chen L , Li J , Dong X , et al . ShareGPT4Video: Improving video understanding and generation with better captions [J ] . arXiv preprint arXiv: 2406.01463 , 2024 .

Wang S , Xu T , Li H , et al . Large language models for education: A survey and outlook [J ] . IEEE Signal Processing Magazine , 2026 , 42 ( 6 ): 51 - 63 .

Li W , Wang F , Mayer R E . How to guide learners’ processing of multimedia lessons with pedagogical agents [J ] . Learning and Instruction , 2023 , 84 : 101729 .

Carlotto T , Jaques P A . The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment [J ] . International Journal of Human-Computer Studies , 2016 , 95 : 15 - 26 .

Li W , Wang F , Mayer R E , et al . Animated pedagogical agents enhance learning outcomes and brain activity during learning [J ] . Journal of Computer Assisted Learning , 2022 , 38 ( 3 ): 621 - 637 .

Davis R O . The impact of pedagogical agent gesturing in multimedia learning environments: A meta-analysis [J ] . Educational Research Review , 2018 , 24 : 193 - 209 .

Chung J S , Zisserman A . Out of time: automated lip sync in the wild [C ] // Asian conference on computer vision . Cham : Springer International Publishing , 2016 : 251 - 263 .

Bai J , Bai S , Yang S , et al . Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond [J ] . arXiv preprint arXiv: 2308.12966 , 2023 .

Bai S , Chen K , Liu X , et al . Qwen2.5-VL Technical Report [J ] . arXiv preprint arXiv: 2502.13923 , 2025 .

Xu J , Guo Z , He J , et al . Qwen2.5-Omni Technical Report [J ] . arXiv preprint arXiv: 2503.20215 , 2025 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A cost-benefit collaborative optimization method for high-quality data preparation

Ontology-driven construction and governance of multimodal datasets for telecommunications

Construction and leakage-controlled evaluation of an airborne multi-source microphysical dataset for weather modification

Research on legal risks and regulatory paths of enterprise data crawling — from the perspective of balancing rights and interests

Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology

Related Author

Ding Xiaoou

Cui Wenxuan

Cheng Haifeng

Qian Zekai

Li Yong

Wang Hongzhi

Wang Chen

Zhang Yawei

Related Institution

National Engineering Research Center for Big Data Software, Tsinghua University

Faculty of Computing, Harbin Institute of Technology

General Office of Harbin Institute of Technology

China Unicom Software Research Institute

Business-Intelligence of Oriental Nations Corporation Ltd.

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰