Full Text:   <240>

Summary:  <17>

CLC number: TP391

On-line Access: 2022-09-21

Received: 2022-03-16

Revision Accepted: 2022-09-21

Crosschecked: 2022-07-28

Cited: 0

Clicked: 226

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Jinxiao XIAO

https://orcid.org/0000-0002-9666-2943

Yansong LI

https://orcid.org/0000-0002-0749-4344

Yun TIAN

https://orcid.org/0000-0001-5574-2325

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2022 Vol.23 No.9 P.1324-1337

http://doi.org/10.1631/FITEE.2200102


Visual recognition of cardiac pathology based on 3D parametric model reconstruction


Author(s):  Jinxiao XIAO, Yansong LI, Yun TIAN, Dongrong XU, Penghui LI, Shifeng ZHAO, Yunhe PAN

Affiliation(s):  School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China; more

Corresponding email(s):   xiaojinxiao@mail.bnu.edu.cn, liyansong@mail.bnu.edu.cn, tianyun@bnu.edu.cn

Key Words:  3D visual knowledge, 3D parametric model, Cardiac pathology diagnosis, Data augmentation


Jinxiao XIAO, Yansong LI, Yun TIAN, Dongrong XU, Penghui LI, Shifeng ZHAO, Yunhe PAN. Visual recognition of cardiac pathology based on 3D parametric model reconstruction[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(9): 1324-1337.

@article{title="Visual recognition of cardiac pathology based on 3D parametric model reconstruction",
author="Jinxiao XIAO, Yansong LI, Yun TIAN, Dongrong XU, Penghui LI, Shifeng ZHAO, Yunhe PAN",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="23",
number="9",
pages="1324-1337",
year="2022",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2200102"
}

%0 Journal Article
%T Visual recognition of cardiac pathology based on 3D parametric model reconstruction
%A Jinxiao XIAO
%A Yansong LI
%A Yun TIAN
%A Dongrong XU
%A Penghui LI
%A Shifeng ZHAO
%A Yunhe PAN
%J Frontiers of Information Technology & Electronic Engineering
%V 23
%N 9
%P 1324-1337
%@ 2095-9184
%D 2022
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2200102

TY - JOUR
T1 - Visual recognition of cardiac pathology based on 3D parametric model reconstruction
A1 - Jinxiao XIAO
A1 - Yansong LI
A1 - Yun TIAN
A1 - Dongrong XU
A1 - Penghui LI
A1 - Shifeng ZHAO
A1 - Yunhe PAN
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 23
IS - 9
SP - 1324
EP - 1337
%@ 2095-9184
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2200102


Abstract: 
Visual recognition of cardiac images is important for cardiac pathology diagnosis and treatment. Due to the limited availability of annotated datasets, traditional methods usually extract features directly from two-dimensional slices of three-dimensional (3D) heart images, followed by pathological classification. This process may not ensure the overall anatomical consistency in 3D heart. A new method for classification of cardiac pathology is therefore proposed based on 3D parametric model reconstruction. First, 3D heart models are reconstructed based on multiple 3D volumes of cardiac imaging data at the end-systole (ES) and end-diastole (ED) phases. Next, based on these reconstructed 3D hearts, 3D parametric models are constructed through the statistical shape model (SSM), and then the heart data are augmented via the variation in shape parameters of one 3D parametric model with visual knowledge constraints. Finally, shape and motion features of 3D heart models across two phases are extracted to classify cardiac pathology. Comprehensive experiments on the automated cardiac diagnosis challenge (ACDC) dataset of the Statistical Atlases and Computational Modelling of the Heart (STACOM) workshop confirm the superior performance and efficiency of this proposed approach.

基于三维参数模型重建的心脏病理视觉识别

肖金肖1,李岩松1,田沄1,徐冬溶2,3,李鹏辉1,赵世凤1,潘云鹤3
1北京师范大学人工智能学院,中国北京市,100875
2哥伦比亚大学精神病学系和纽约州立精神病学研究所,美国纽约市,10032
3浙江大学计算机科学与技术学院,中国杭州市,310027
摘要:心脏图像的视觉识别对于心脏病理诊断和治疗具有重要意义。由于可用标注数据集有限,传统方法通常基于三维心脏图像的二维切片对病理分类特征进行提取,难以确保心脏解剖结构的整体一致性。为此,本文提出一种基于三维参数模型重建的心脏病理分类方法。首先,基于收缩末期和舒张末期时相心脏图像的多个三维心脏成像数据重建三维心脏模型。其次,基于重建的三维心脏模型,通过统计形状模型方法构建三维参数模型。然后,基于三维统计形状模型及其视觉知识约束对心脏数据进行增强。最后,提取不同时相的三维心脏模型的形状和运动特征,对心脏病理进行分类。在STACOM公开挑战赛的ACDC数据集上的实验验证了所提方法的优越性和有效性。

关键词:三维视觉知识;三维参数模型;心脏病理诊断;数据增强

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Ammar A, Bouattane O, Youssfi M, 2021. Automatic cardiac cine MRI segmentation and heart disease classification. Comput Med Imag Graph, 88:101864.

[2]Attar R, Pereañez M, Bowles C, et al., 2019. 3D cardiac shape prediction with deep neural networks: simultaneous use of images and patient metadata. Proc 22nd Int Conf on Medical Image Computing and Computer-Assisted Intervention, p.586-594.

[3]Bai WJ, Oktay O, Rueckert D, 2016. Classification of myocardial infarcted patients by combining shape and motion features. Proc 6th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.140-145.

[4]Bernard O, Lalande A, Zotti C, et al., 2018. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imag, 37(11):2514-2525.

[5]Bernardino G, Benkarim O, Sanz-de la Garza M, et al., 2020. Handling confounding variables in statistical shape analysis-application to cardiac remodelling. Med Image Anal, 65:101792.

[6]Besl PJ, McKay ND, 1992. A method for registration of 3-D shapes. IEEE Trans Patt Anal Mach Intell, 14(2):239-256.

[7]Biffi C, Cerrolaza JJ, Tarroni G, et al., 2019. 3D high-resolution cardiac segmentation reconstruction from 2D views using conditional variational autoencoders. Proc IEEE 16th Int Symp on Biomedical Imaging, p.1643-1646.

[8]Brock A, Donahue J, Simonyan K, 2019. Large scale GAN training for high fidelity natural image synthesis. https://arxiv.org/abs/1809.11096

[9]Cerqueira MD, Weissman NJ, Dilsizian V, et al., 2002. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart—a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation, 105(4):539-542.

[10]Cetin I, Sanroma G, Petersen SE, et al., 2017. A radiomics approach to computer-aided diagnosis with cardiac cine-MRI. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.82-90.

[11]Chang YK, Jung C, 2020. Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds. Neurocomputing, 418:270-279.

[12]Charles RQ, Su H, Mo KC, et al., 2017. PointNet: deep learning on point sets for 3D classification and segmentation. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.77-85.

[13]Chen X, Ravikumar N, Xia Y, et al., 2021. Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds. Med Imag Anal, 74:102228.

[14]Cootes TF, Taylor CJ, Cooper DH, et al., 1995. Active shape models—their training and application. Comput Vis Imag Underst, 61(1):38-59.

[15]Dryden IL, Mardia KV, 1998. Statistical Shape Analysis. John Wiley & Sons, London, UK, p.663-669.

[16]Frangi AF, Rueckert D, Schnabel JA, et al., 2002. Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling. IEEE Trans Med Imag, 21(9):1151-1166.

[17]Gao ZF, Zhang HY, Wang DF, et al., 2018. Robust recovery of myocardial kinematics using dual H criteria. Multim Tools Appl, 77(17):23043-23071.

[18]Gjesdal O, Bluemke DA, Lima JA, 2011. Cardiac remodeling at the population level—risk factors, screening, and outcomes. Nat Rev Cardiol, 8(12):673-685.

[19]Isensee F, Jaeger PF, Full PM, et al., 2018. Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.120-129.

[20]Jolliffe IT, 2002. Principal component analysis. J Mark Res, 87(4):513.

[21]Karras T, Laine S, Aila T, 2019. A style-based generator architecture for generative adversarial networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4396-4405.

[22]Khened M, Alex V, Krishnamurthi G, 2018. Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.140-151.

[23]Liu HF, Shi PC, 2009. Maximum a posteriori strategy for the simultaneous motion and material property estimation of the heart. IEEE Trans Biomed Eng, 56(2):378-389.

[24]Pan YH, 1996. The synthesis reasoning. Patt Recogn Artif Intell, 9(3):201-208 (in Chinese).

[25]Pan YH, 2019. On visual knowledge. Front Inform Technol Electron Eng, 20(8):1021-1025.

[26]Pan YH, 2020. Multiple knowledge representation of artificial intelligence. Engineering, 6(3):216-217.

[27]Pan YH, 2021a. Miniaturized five fundamental issues about visual knowledge. Front Inform Technol Electron Eng, 22(5):615-618.

[28]Pan YH, 2021b. On visual understanding. Front Inform Technol Electron Eng, early access.

[29]Parajuli N, Lu A, Duncan JS, 2016. Left ventricle classification using active shape model and support vector machine. Proc 6th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.154-161.

[30]Rodero C, Strocchi M, Marciniak M, et al., 2021. Linking statistical shape models and simulated function in the healthy adult human heart. PLoS Comput Biol, 17(4):e1008851.

[31]Suinesiaputra A, Ablin P, Albà X, et al., 2018. Statistical shape modeling of the left ventricle: myocardial infarct classification challenge. IEEE J Biomed Health Inform, 22(2):503-515.

[32]Thermos S, Liu X, O’Neil A, et al., 2021. Controllable cardiac synthesis via disentangled anatomy arithmetic. Proc 24th Int Conf on Medical Image Computing and Computer-Assisted Intervention, p.160-170.

[33]Van Dijck C, Wirix-Speetjens R, Jonkers I, et al., 2018. Statistical shape model-based prediction of tibiofemoral cartilage. Comput Methods Biomech Biomed Eng, 21(9):568-578.

[34]Wolterink JM, Leiner T, Viergever MA, et al., 2018. Automatic segmentation and disease classification using cardiac cine MR images. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.101-110.

[35]Zheng Q, Delingette H, Ayache N, 2019. Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow. Med Imag Anal, 56:80-95.

[36]Zhou QY, Park J, Koltun V, 2016. Fast global registration. Proc 14th European Conf on Computer Vision, p.766-782.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE