Full Text:   <2161>

Summary:  <82>

CLC number: TP391.4

On-line Access: 2022-03-22

Received: 2020-09-28

Revision Accepted: 2021-04-15

Crosschecked: 0000-00-00

Cited: 0

Clicked: 4248

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Yuan HUANG

https://orcid.org/0000-0002-2755-0550

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2022 Vol.23 No.3 P.398-408

http://doi.org/10.1631/FITEE.2000508


Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline


Author(s):  Yuan HUANG, Feipeng DA

Affiliation(s):  School of Automation, Southeast University, Nanjing 210096, China; more

Corresponding email(s):   whhbb@163.com

Key Words:  Three-dimensional (3D) point cloud, Hole filling, Stereo matching, B-spline


Yuan HUANG, Feipeng DA. Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline[J]. Frontiers of Information Technology & Electronic Engineering, 2022, 23(3): 398-408.

@article{title="Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline",
author="Yuan HUANG, Feipeng DA",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="23",
number="3",
pages="398-408",
year="2022",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2000508"
}

%0 Journal Article
%T Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline
%A Yuan HUANG
%A Feipeng DA
%J Frontiers of Information Technology & Electronic Engineering
%V 23
%N 3
%P 398-408
%@ 2095-9184
%D 2022
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2000508

TY - JOUR
T1 - Three-dimensional face point cloud hole-filling algorithm based on binocular stereo matching and a B-spline
A1 - Yuan HUANG
A1 - Feipeng DA
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 23
IS - 3
SP - 398
EP - 408
%@ 2095-9184
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2000508


Abstract: 
When obtaining three-dimensional (3D) face point cloud data based on structured light, factors related to the environment, occlusion, and illumination intensity lead to holes in the collected data, which affect subsequent recognition. In this study, we propose a hole-filling method based on stereo-matching technology combined with a b-spline. The algorithm uses phase information acquired during raster projection to locate holes in the point cloud, simultaneously extracting boundary point cloud sets. By registering the face point cloud data using the stereo-matching algorithm and the data collected using the raster projection method, some supplementary information points can be obtained at the holes. The shape of the b-spline curve can then be roughly described by a few key points, and the control points are put into the hole area as key points for iterative calculation of surface reconstruction. Simulations using smooth ceramic cups and human face models showed that our model can accurately reproduce details and accurately restore complex shapes on the test surfaces. Simulation results indicated the robustness of the method, which is able to fill holes on complex areas such as the inner side of the nose without a prior model. This approach also effectively supplements the hole information, and the patched point cloud is closer to the original data. This method could be used across a wide range of applications requiring accurate facial recognition.

基于双目立体匹配和B样条的三维人脸点云孔洞修补算法

黄源1,2,达飞鹏1,2
1东南大学自动化学院,中国南京市,210096
2东南大学深圳研究院,中国深圳市,518000
摘要:在基于结构光的三维人脸点云数据采集过程中,由于环境、遮挡以及光照强度等因素影响,采集到的数据往往会出现孔洞区域,从而影响后续识别效果。本文提出一种采用立体匹配技术结合B样条的孔洞修补方法。算法首先利用光栅投影过程中获取的相位信息定位点云中的孔洞区域,同时提取边界点集。然后将立体匹配算法获取的人脸点云数据同光栅投影法采集的数据进行配准,在孔洞处选取初始修补控制点。再利用B样条曲线形状可由少数关键点大致描述这一特性,将控制点作为关键点放入孔洞区域进行曲面重建迭代计算。仿真使用光滑陶瓷杯和人脸模型进行,结果表明,该算法能够准确再现被测物体表面的细节和复杂形状。同时也说明所提方法具有强鲁棒性,能够在完全无先验信息的情况下对物体复杂区域实现孔洞修补,并且修补后的点云更加接近原始数据。该方法可广泛应用于需要精确人脸识别的领域。

关键词:三维点云;孔洞修补;立体匹配;B样条

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]BendelsGH, SchnabelR, KleinR, 2006. Detecting holes in point set surfaces. J WSCG, 14:89-96.

[2]BlackJAJr., GargeshaM, KaholK, et al., 2016. Framework for performance evaluation of face recognition algorithms. SPIE 4862:163-174.

[3]BuchinK, van KreveldM, MeijerH, et al., 2009. On planar supports for hypergraphs. Proc 17th Int Conf on Graph Drawing, p.345-356.

[4]CarrJC, BeatsonRK, CherrieJB, et al., 2001. Reconstruction and representation of 3D objects with radial basis functions. Proc 28th Annual Conf on Computer Graphics and Interactive Techniques, p.67-76.

[5]ChenH, MaSW, NuechterA, 2016. Non-synchronous point cloud algorithm for 3D reconstruction based on laser scanning and SFM. Chin J Sci Instrum, 37(5):1148-1157 (in Chinese).

[6]ChuiCK, LaiMJ, 2000. Filling polygonal holes using C1 cubic triangular spline patches. Comput Aided Geom Des, 17(4):297-307.

[7]FloaterMS, ReimersM, 2001. Meshless parameterization and surface reconstruction. Comput Aided Geom Des, 18(2): 77-92.

[8]FurukawaY, PonceJ, 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans Patt Anal Mach Intell, 32(8):1362-1376.

[9]GilaniZS, MianA, 2016. Towards large-scale 3D face recognition. Proc Int Conf on Digital Image Computing: Techniques and Applications, p.1-8.

[10]GoeseleM, CurlessB, SeitzSM, 2006. Multi-view stereo revisited. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.2402-2409.

[11]HuangXS, ZhangJ, FanLX, et al., 2017. A systematic approach for cross-source point cloud registration by preserving macro and micro structures. IEEE Trans Image Process, 26(7):3261-3276.

[12]HuangY, DaFP, 2019. Registration algorithm for point cloud based on normalized cross-correlation. IEEE Access, 7:137136-137146.

[13]HuangY, DaFP, TaoHJ, 2015. An automatic registration algorithm for point cloud based on feature extraction. Chin J Lasers, 42(3):308002 (in Chinese).

[14]JeongY, BokY, KimJS, et al., 2011. Complementation of cameras and lasers for accurate 6D SLAM: from correspondences to bundle adjustment. Proc IEEE Int Conf on Robotics and Automation, p.3581-3588.

[15]JunY, 2005. A piecewise hole filling algorithm in reverse engineering. Comput-Aided Des, 37(2):263-270.

[16]KurlinV, 2014. A fast and robust algorithm to count topologically persistent holes in noisy clouds. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1458-1463.

[17]LiuK, ZhouCH, WeiSB, et al., 2014. Optimized stereo matching in binocular three-dimensional measurement system using structured light. Appl Opt, 53(26):6083-6090.

[18]LiuYJ, WangM, ZhangHL, et al., 2016. Strategy of classification and repairing for hole of incomplete point clouds based on fuzzy inference. J Comput Theor Nanosci, 13 (11):8227-8233.

[19]NguyenVS, TrinhTH, TranMH, 2015. Hole boundary detection of a surface of 3D point clouds. Proc Int Conf on Advanced Computing and Applications, p.124-129.

[20]OrriolsX, BinefaX, 2003. Finding breaking curves in 3D surface. Proc 1st Iberian Conf on Pattern Recognition and Image Analysis, p.681-688.

[21]O’TooleAJ, AnXB, DunlopJ, et al., 2012. Comparing face recognition algorithms to humans on challenging tasks. ACM Trans Appl Percept, 9(4):16.

[22]PanYH, 2019. On visual knowledge. Front Inform Technol Electron Eng, 20(8):1021-1025.

[23]PanYH, 2021. Miniaturized five fundamental issues about visual knowledge. Front Inform Technol Electron Eng, 22(5):615-618.

[24]PanchettiM, PernotJP, VéronP, 2010. Towards recovery of complex shapes in meshes using digital images for reverse engineering applications. Comput-Aided Des, 42(8):693-707.

[25]PernotJP, MoraruG, VéronP, 2007. Repairing triangle meshes built from scanned point cloud. J Eng Des, 18(5): 459-473.

[26]QuinsatY, LartigueC, 2015. Filling holes in digitized point cloud using a morphing-based approach to preserve volume characteristics. Int J Adv Manuf Technol, 81(1-4): 411-421.

[27]Russell, StuartJ, NorvigP, 2010. Artificial intelligence: a modern approach. Appl Mech Mater, 263(5):2829-2833.

[28]SchafferM, GrosseM, HarendtB, et al., 2011. High-speed three-dimensional shape measurements of objects with laser speckles and acousto-optical deflection. Opt Lett, 36(16):3097-3099.

[29]ShiLM, GuoFS, HuZY, 2011. An improved PMVS through scene geometric information. Acta Autom Sin, 37(5):560-568 (in Chinese).

[30]StoneEE, SkubicM, 2015. Fall detection in homes of older adults using the Microsoft Kinect. IEEE J Biomed Health Inform, 19(1):290-301.

[31]WangJN, OliveiraMM, 2007. Filling holes on locally smooth surfaces reconstructed from point clouds. Image Vis Comput, 25(1):103-113.

[32]ZhengEL, WuCC, 2015. Structure from motion using structure-less resection. Proc IEEE Int Conf on Computer Vision, p.2075-2083.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE