|
Frontiers of Information Technology & Electronic Engineering
ISSN 2095-9184 (print), ISSN 2095-9230 (online)
2023 Vol.24 No.10 P.1482-1496
Path guided motion synthesis for Drosophila larvae
Abstract: The deformability and high degree of freedom of mollusks bring challenges in mathematical modeling and synthesis of motions. Traditional analytical and statistical models are limited by either rigid skeleton assumptions or model capacity, and have difficulty in generating realistic and multi-pattern mollusk motions. In this work, we present a large-scale dynamic pose dataset of Drosophila larvae and propose a motion synthesis model named Path2Pose to generate a pose sequence given the initial poses and the subsequent guiding path. The Path2Pose model is further used to synthesize long pose sequences of various motion patterns through a recursive generation method. Evaluation analysis results demonstrate that our novel model synthesizes highly realistic mollusk motions and achieves state-of-the-art performance. Our work proves high performance of deep neural networks for mollusk motion synthesis and the feasibility of long pose sequence synthesis based on the customized body shape and guiding path.
Key words: Motion synthesis of mollusks; Dynamic pose dataset; Morphological analysis; Long pose sequence generation
1之江实验室基础理论研究院,中国杭州市,311121
2康复大学康复科学与工程学院,中国青岛市,266114
3浙江大学求是高等研究院,中国杭州市,310027
4浙江大学医学院附属精神卫生中心附属第二医院神经生物学与神经病学系,中国杭州市,310058
5浙江大学脑科学与脑医学学院教育部脑与脑机融合前沿科学中心,医学神经生物学卫生部重点实验室,中国杭州市,310058
摘要:软体动物身体可变形性和高自由度的特点为数学建模和运动合成带来很大挑战。受限于刚体骨骼假设或模型容量,传统解析模型和统计模型难以生成逼真和多模态的软体动物运动。本文建立一个大规模果蝇幼虫动态姿态数据集,并提出一个运动合成模型(Path2Pose),通过给定一段幼虫初始运动姿态序列和引导路径生成后续运动姿态序列。进一步地,通过循环生成的方式,Path2Pose模型可以合成长时间、多模态的果蝇幼虫运动姿态序列。运动评估实验表明,Path2Pose模型可以生成高度真实的软体动物运动,并在现有同类型模型中取得最好生成效果。本文的工作证明了深度神经网络在软体动物运动合成任务中的良好性能以及通过定制软体动物体型和引导路径生成长时间运动姿态的可行性。
关键词组:
References:
Open peer comments: Debate/Discuss/Question/Opinion
<1>
DOI:
10.1631/FITEE.2200529
CLC number:
Q811.211
Download Full Text:
Downloaded:
4752
Download summary:
<Click Here>Downloaded:
376Clicked:
1533
Cited:
0
On-line Access:
2024-08-27
Received:
2023-10-17
Revision Accepted:
2024-05-08
Crosschecked:
2023-10-27