Journal of Zhejiang University

Journal of Zhejiang University SCIENCE B 2026 Vol.27 No.9 P.946-960

Three-dimensional face reconstruction for emotional dynamics: a novel approach to distinguishing bipolar disorder from major depressive disorder

Author(s): Tao DU,Jinchao GE,Hong LYU,Yutong ZHANG,Xin XU,Shaohua HU,Jingkai CHEN
Affiliation(s): 1. Department of Psychiatry, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou 310003, China more
Corresponding email(s): xuxinsummer@zju.edu.cn, dorhushaohua@zju.edu.cn, wzcjk922@163.com
Key Words: Major depressive disorder, Bipolar disorder, 3D facial reconstruction, Emotion dynamics, Valence and arousal

<<< Previous Article \|Next Article >>>

Tao DU, Jinchao GE, Hong LYU, Yutong ZHANG, Xin XU, Shaohua HU, Jingkai CHEN. Three-dimensional face reconstruction for emotional dynamics: a novel approach to distinguishing bipolar disorder from major depressive disorder[J]. Journal of Zhejiang University Science B, 2026, 27(9): 946-960.

@article{title="Three-dimensional face reconstruction for emotional dynamics: a novel approach to distinguishing bipolar disorder from major depressive disorder",
author="Tao DU, Jinchao GE, Hong LYU, Yutong ZHANG, Xin XU, Shaohua HU, Jingkai CHEN",
journal="Journal of Zhejiang University Science B",
volume="27",
number="9",
pages="946-960",
year="2026",
publisher="Zhejiang University Press & Springer",
doi="10.1631/jzus.B2500705"
}

%0 Journal Article
%T Three-dimensional face reconstruction for emotional dynamics: a novel approach to distinguishing bipolar disorder from major depressive disorder
%A Tao DU
%A Jinchao GE
%A Hong LYU
%A Yutong ZHANG
%A Xin XU
%A Shaohua HU
%A Jingkai CHEN
%J Journal of Zhejiang University SCIENCE B
%V 27
%N 9
%P 946-960
%@ 1673-1581
%D 2026
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.B2500705

TY - JOUR
T1 - Three-dimensional face reconstruction for emotional dynamics: a novel approach to distinguishing bipolar disorder from major depressive disorder
A1 - Tao DU
A1 - Jinchao GE
A1 - Hong LYU
A1 - Yutong ZHANG
A1 - Xin XU
A1 - Shaohua HU
A1 - Jingkai CHEN
J0 - Journal of Zhejiang University Science B
VL - 27
IS - 9
SP - 946
EP - 960
%@ 1673-1581
Y1 - 2026
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.B2500705

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: Major depressive disorder (MDD) and bipolar disorder (BD) are frequently misdiagnosed in clinical practice due to their overlapping symptoms, resulting in delayed treatment and an increased burden on patients. Reliable biomarkers for early differential diagnosis are currently lacking. To address this, we developed a clinical facial video dataset of patients with BD and MDD and introduced an adaptive-shape vision-to-language model (AsmV2L), a vision-language-guided fine-grained three-dimensional (3D) face reconstruction framework that better preserves affect-related facial cues from monocular videos. Using valence-arousal (VA) representations estimated from these reconstructed 3D faces, we identified distinct affective patterns that differentiated the two disorders. In the subject-level downstream evaluation, the emotional dynamic features extracted by our framework achieved 88.9% accuracy in distinguishing BD from MDD. Additionally, experiments on public benchmark datasets demonstrated that AsmV2L maintained competitive 3D reconstruction accuracy while preserving emotion-related semantic information more effectively.

面向情感动态的三维人脸重建：一种区分双相情感障碍与重性抑郁障碍的新方法

杜涛^1,3, 葛金超², 吕泓^1,3, 张毓桐^1,4, 徐欣^5,6,7, 胡少华^1,6,7,9, 陈京凯^1,8
¹浙江大学医学院附属第一医院精神科，中国杭州， 310003
²沃隆港大学计算与信息技术学院，澳大利亚新南威尔士州沃隆港， 2522
³沈阳理工大学自动化与电气工程学院，中国沈阳， 110159
⁴拉夫堡大学计算机科学系，英国拉夫堡， LE11 3TU
⁵浙江大学医学院附属第二医院公共卫生学院，中国杭州， 310058
⁶南湖脑机接口研究院，中国杭州， 311100
⁷浙江大学医学院脑机智能全国重点实验室，脑科学与脑机融合教育部前沿科学中心，中国杭州， 310003
⁸浙江省精准精神病学重点实验室，中国杭州， 310003
⁹浙江大学医学院良渚实验室，中国杭州， 310000
摘要：重度抑郁障碍（major depressive disorder，MDD）与双相情感障碍（bipolar disorder，BD）由于临床症状存在重叠，在临床实践中容易发生误诊，从而导致治疗延误并增加患者负担。目前，针对这两种疾病进行早期鉴别诊断的可靠生物标志物仍较为缺乏。为解决这一问题，本研究构建了由BD和MDD患者组成的临床面部视频数据集，并提出视觉-语言自适应形状模型（adaptive-shape vision-to-language model，AsmV2L）。该模型是一种视觉-语言引导的细粒度三维（three-dimensional，3D）人脸重建框架，能够更好地保留单目视频中与情感相关的面部线索。基于重建三维人脸所估计的效价-唤醒度（valence-arousal，VA）表征，本研究识别出了可用于区分两种疾病的特异性情感模式。在受试者水平的下游评估中，本框架提取的情绪动态特征在区分BD与MDD时取得了88.9%的准确率。此外，在公共基准数据集上的实验结果表明，AsmV2L在保持具有竞争力的三维人脸重建精度的同时，能够更加有效地保留与情绪相关的语义信息。

关键词：重度抑郁障碍；双相情感障碍；三维人脸重建；情绪动态；效价和唤醒度

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Aneja S, Thies J, Dai A, et al., 2023. ClipFace: text-guided editing of textured 3D morphable models. SIGGRAPH '23 Conference Proceedings, August 6‒10, 2023, Los Angeles, CA, USA, Article No. 70, p.1-11.

[2]Cen MY, Shen HR, Zhao WY, et al., 2024. A survey of text-guided 3D face reconstruction. 2024 3rd International Conference on Image Processing and Media Computing (ICIPMC), Hefei, China, p.82-87.

[3]Chai ZH, Zhang HX, Ren J, et al., 2022. REALY: rethinking the evaluation of 3D face reconstruction. In: Avidan S, Brostow G, Cissé M, et al. (Eds.), Computer Vision ‒ ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, Vol. 13668. Springer, Cham, p.74-92.

[4]Chen L, Xu YY, Lin JY, et al., 2024. The prevalence and clinical correlates of suicide attempts in patients with bipolar disorder misdiagnosed with major depressive disorder: results from a national survey in China. Asian J Psychiatr, 93:103958.

[5]Chen XN, Lu YM, Cue JM, et al., 2025. Classification of schizophrenia, bipolar disorder and major depressive disorder with comorbid traits and deep learning algorithms. Schizophrenia, 11:14.

[6]Dai CX, Fu YY, Li XW, et al., 2025. Clinical efficacy and safety of vortioxetine as an adjuvant drug for patients with bipolar depression. J Zhejiang Univ-Sci B (Biomed & Biotechnol), 26(1):26-38.

[7]Daněček R, Black M, Bolkart T, 2022. EMOCA: emotion driven monocular face capture and animation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, p.20279-20290.

[8]Deng Y, Yang JL, Xu SC, et al., 2019. Accurate 3D face reconstruction with weakly-supervised learning: from single image to image set. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, p.285-295.

[9]Dong L, Wang X, Setlur S, et al., 2025. Ig3D: integrating 3D face representations in facial expression inference. In: del Bue A, Canton C, Pont-Tuset J, et al. (Eds.), Computer Vision ‒ ECCV 2024 Workshops. ECCV 2024. Lecture Notes in Computer Science, Vol. 15637. Springer, Cham, p.404-421.

[10]Fagiolini A, Florea I, Loft H, et al., 2021. Effectiveness of vortioxetine on emotional blunting in patients with major depressive disorder with inadequate response to SSRI/SNRI treatment. J Affect Disord, 283:472-479.

[11]Feng Y, Feng HW, Black MJ, et al., 2021. Learning an animatable detailed 3D face model from in-the-wild images. ACM Trans Graph, 40(4):88.

[12]Hu RG, Wang XK, Zhao CR, 2025. Identity aware 3D face reconstruction from in-the-wild images. Neurocomputing, 641:130299.

[13]Jahromi GG, Rezaei N, 2025. Connecting the dots: NLRP3 inflammasome as a key mediator in the intersection of depression and cardiovascular disease ‒ a narrative review. Heart Mind, 9(1):48-60.

[14]Jiang XW, Cao B, Li C, et al., 2024. Identifying misdiagnosed bipolar disorder using support vector machine: feature selection based on fMRI of follow-up confirmed affective disorders. Transl Psychiatry, 14:9.

[15]Jin RF, Zhou RH, Zhang D, 2025. Recent advances in antibody optimization based on deep learning methods. J Zhejiang Univ-Sci B (Biomed & Biotechnol), 26(5):409-420.

[16]Kang XY, Liu XY, Chen ST, et al., 2024. Major depressive disorder recognition by quantifying EEG signal complexity using proposed APLZC and AWPLZC. J Affect Disord, 356:105-114.

[17]Karras T, Laine S, Aila T, 2019. A style-based generator architecture for generative adversarial networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, p.4396-4405.

[18]Kollias D, Zafeiriou S, 2021. Affect analysis in-the-wild: valence-arousal, expressions, action units and a unified framework. arXiv:2103.15792.

[19]Kollias D, Tzirakis P, Baird A, et al., 2023. ABAW: valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada, p.5889-5898.

[20]Li TY, Bolkart T, Black MJ, et al., 2017. Learning a model of facial shape and expression from 4D scans. ACM Trans Graph, 36(6):194.

[21]Liu ZW, Luo P, Wang XG, et al., 2015. Deep learning face attributes in the wild. 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, p.3730-3738.

[22]Loosen AM, Kato A, Gu XS, 2025. Revisiting the role of computational neuroimaging in the era of integrative neuroscience. Neuropsychopharmacology, 50:103-113.

[23]Martikkala A, Baryshnikov I, Granroth-Wilding H, et al., 2025. Temporal variations of depressive symptoms in patients with bipolar, borderline personality, and major depressive disorder: an ecological momentary assessment study. J Psychiatr Res, 191:313-322.

[24]Mollahosseini A, Hasani B, Mahoor MH, 2019. AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans Affect Comput, 10(1):18-31.

[25]Preller KH, Scholpp J, Wunder A, et al., 2024. Neuroimaging biomarkers for drug discovery and development in schizophrenia. Biol Psychiatry, 96(8):666-673.

[26]R MB, Tewari A, Seidel HP, et al., 2021. Learning complete 3D morphable face models from images and videos. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, p.3360-3370.

[27]Retsinas G, Filntisis PP, Daněček R, et al., 2024. 3D facial expressions through analysis-by-neural-synthesis. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, p.2490-2501.

[28]Sadeghi M, Richer R, Egger B, et al., 2024. Harnessing multimodal approaches for depression detection using large language models and facial expressions. npj Ment Health Res, 3:66.

[29]Sarmin N, Roknuzzaman ASM, Mouree TZ, et al., 2024. Evaluation of serum interleukin-12 and interleukin-4 as potential biomarkers for the diagnosis of major depressive disorder. Sci Rep, 14:1652.

[30]Schneider H, Pavlitska S, Gremmelmaier H, et al., 2025. Datasets for valence and arousal inference: a survey. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, p.5657-5664.

[31]Shi B, Yu BW, Chen C, et al., 2025. Guidelines for cognitive clinical diagnosis and treatment of coronary artery disease complicated with depression and anxiety: December 2025 update. Heart Mind, 9(6):462-471.

[32]Singh B, Swartz HA, Cuellar-Barboza AB, et al., 2025. Bipolar disorder. Lancet, 406(10506):963-978.

[33]Steardo L, D’Angelo M, Monaco F, et al., 2025. Decoding neural circuit dysregulation in bipolar disorder: toward an advanced paradigm for multidimensional cognitive, emotional, and psychomotor treatment. Neurosci Biobehav Rev, 169:106030.

[34]Sun JX, Wang X, Wang LZ, et al., 2023. Next3D: generative neural texture rasterization for 3D-aware head avatars. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, p.20991-21002.

[35]Tellamekala MK, Sümer Ö, Schuller BW, et al., 2024. Are 3D face shapes expressive enough for recognising continuous emotions and action unit intensities? IEEE Trans Affect Comput, 15(2):535-548.

[36]Wang JT, Yu CC, Li HB, 2024. Multi-modal feature guided detailed 3D face reconstruction from a single image. In: Liu Q, Wang H, Ma Z, et al. (Eds.), Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, Vol. 14426. Springer, Singapore, p.356-368.

[37]Wang P, Bai YL, Xiao Y, et al., 2025. Aberrant network topological structure of sensorimotor superficial white-matter system in major depressive disorder. J Zhejiang Univ-Sci B (Biomed & Biotechnol), 26(1):39-51.

[38]Wang ZD, Zhu XY, Zhang TS, et al., 2024. 3D face reconstruction with the geometric guidance of facial part segmentation. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, p.1672-1682.

[39]Wu YJ, Meng YP, Hu ZP, et al., 2024. Text-guided 3D face synthesis ‒ from generation to editing. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, p.1260-1269.

[40]Yin GM, Yuan J, Chen YJ, et al., 2025. Schizophrenia recognition based on three-dimensional adaptive graph convolutional neural network. Sci Rep, 15:4067.

[41]Younis EMG, Mohsen S, Houssein EH, et al., 2024. Machine learning for human emotion recognition: a comprehensive review. Neural Comput Appl, 36(16):8901-8947.

[42]Zhang QT, Feng JQ, 2025. Disentangled geometry and appearance for efficient multi-view surface reconstruction and rendering. arXiv:2508.17436.

[43]Zheng YF, Wang YF, Wetzstein G, et al., 2023. PointAvatar: deformable point-based head avatars from videos. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, p.21057-21067.

[44]Zhu CA, Joslin C, 2024. A review of motion retargeting techniques for 3D character facial animation. Comput Graphics, 123:104037.

[45]Zielonka W, Bolkart T, Thies J, 2022. Towards metrical reconstruction of human faces. In: Avidan S, Brostow G, Cissé M, et al. (Eds.), Computer Vision ‒ ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, Vol. 13673. Springer, Cham, p.250-269.

Open peer comments: Debate/Discuss/Question/Opinion

<1>