Full Text:   <2780>

Summary:  <2029>

CLC number: TP391.4

On-line Access: 2024-08-27

Received: 2023-10-17

Revision Accepted: 2024-05-08

Crosschecked: 2016-04-18

Cited: 0

Clicked: 6768

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Chu-hua Huang

http://orcid.org/0000-0003-3507-4256

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering  2016 Vol.17 No.5 P.422-434

http://doi.org/10.1631/FITEE.1500316


A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence


Author(s):  Chu-hua Huang, Dong-ming Lu, Chang-yu Diao

Affiliation(s):  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   diaochangyu@gmail.com

Key Words:  Multi-view video, Free-viewpoint video, Point-pair, Multiscale-contour-based interpolation, Spatio-temporal-contour, Consistency, Time-varying point cloud sequence


Chu-hua Huang, Dong-ming Lu, Chang-yu Diao. A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence[J]. Frontiers of Information Technology & Electronic Engineering, 2016, 17(5): 422-434.

@article{title="A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence",
author="Chu-hua Huang, Dong-ming Lu, Chang-yu Diao",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="17",
number="5",
pages="422-434",
year="2016",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.1500316"
}

%0 Journal Article
%T A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence
%A Chu-hua Huang
%A Dong-ming Lu
%A Chang-yu Diao
%J Frontiers of Information Technology & Electronic Engineering
%V 17
%N 5
%P 422-434
%@ 2095-9184
%D 2016
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.1500316

TY - JOUR
T1 - A multiscale-contour-based interpolation framework for generating a time-varying quasi-dense point cloud sequence
A1 - Chu-hua Huang
A1 - Dong-ming Lu
A1 - Chang-yu Diao
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 17
IS - 5
SP - 422
EP - 434
%@ 2095-9184
Y1 - 2016
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.1500316


Abstract: 
To speed up the reconstruction of 3D dynamic scenes in an ordinary hardware platform, we propose an efficient framework to reconstruct 3D dynamic objects using a multiscale-contour-based interpolation from multi-view videos. Our framework takes full advantage of spatio-temporal-contour consistency. It exploits the property to interpolate single contours, two neighboring contours which belong to the same model, and two contours which belong to the same view at different times, corresponding to point-, contour-, and model-level interpolations, respectively. The framework formulates the interpolation of two models as point cloud transport rather than non-rigid surface deformation. Our framework speeds up the reconstruction of a dynamic scene while improving the accuracy of point-pairing which is used to perform the interpolation. We obtain a higher frame rate, spatio-temporal-coherence, and a quasi-dense point cloud sequence with color information. Experiments with real data were conducted to test the efficiency of the framework.

The proposed method has limited complexity which is good for the intended use (e.g., real-time rendering or at least higher frame rate). On the other hand, the method is still far from being real-time (e.g., 50 seconds/frame) and the degree of scientific innovation is somehow limited as well. On the other hand, the paper is well structured on experiments were performed on real data sets.

基于多尺度轮廓插值生成准密集时变点云模型序列

目的:提高采用多视角视频序列生成时变三维模型序列的速度,以更好地跟踪三维动态物体的形变过程;同时保证时变三维模型序列的外观质量。
创新点:提出基于多尺度轮廓插值生成时变点云模型序列的方法。利用重建对象轮廓的多尺度时空连续性,完成时变三维模型序列的时空维度插值。
方法:首先,采用基于剪影轮廓原理重建物体关键帧的稀疏三维模型。接着,分析三维模型的轮廓点在点级别、轮廓级别、模型级别的连续性,并在该过程中采用距离图插值来增强轮廓的连续性。然后,采用最近点查找方法获得匹配点对,在三个尺度上对匹配点对进行线性密集化。最后,生成具有良好时空一致性的准密集时变三维模型序列。
结论:利用轮廓多尺度时空连续性能够提高重建对象的形变跟踪速度,且时变三维模型序列具有良好的外观质量。

关键词:多视角视频序列;自由视点视频序列;匹配点对;多尺度轮廓插值;时空轮廓;一致性;时变点云模型序列

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Ahmed, N., Junejo, I.N., 2013. A system for 3D video acquisition and spatio-temporally coherent 3D animation reconstruction using multiple RGB-D cameras. Int. J. Signal Process. Image Process. Patt. Recogn., 6(2):113-128.

[2]Allain, B., Franco, J.S., Boye, R.E., 2015. An efficient volumetric framework for shape tracking. IEEE Conf. on Computer Vision and Pattern Recognition, p.268-276.

[3]Arita, D., Taniguchi, R., 2001. RPV-II: a stream-based real-time parallel vision system and its application to real-time volume reconstruction. 2nd Int. Workshop on Computer Vision Systems, p.174-189.

[4]Baumgart, B.G., 1974. Geometric Modeling for Computer Vision. PhD Thesis, Stanford University, Stanford, USA.

[5]Bilir, S.C., Yemez, Y., 2012. Non-rigid 3D shape tracking from multiview video. Comput. Vis. Image Understand., 116(11):1121-1134.

[6]Borovikov, E., Sussman, A., Davis, L., 2003. A high performance multi-perspective vision studio. 17th Annual Int. Conf. on Supercomputing, p.348-357.

[7]Cheung, G.K.M., Kanade, T., Bouguet, J.Y., 2000. A real time system for robust 3D voxel reconstruction of human motions. IEEE Conf. on Computer Vision and Pattern Recognition, p.714-720.

[8]Cheung, G.K.M., Baker, S., Kanade, T., 2003. Visual hull alignment and refinement across time: a 3D reconstruction algorithm combining shape-from-silhouette with stereo. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.375-382.

[9]Díaz-Más, L., Muñoz-Salinas, R., Madrid-Cuevas, F.J., et al., 2010. Shape from silhouette using Dempster-Shafer theory. Patt. Recog., 43(6):2119-2131.

[10]Duckworth, T., Roberts, D.J., 2011. Accelerated polyhedral visual hulls using OpenCL. IEEE Virtual Reality Conf., p.203-204.

[11]Franco, J.S., Boyer, E., 2009. Efficient polyhedral modeling from silhouettes. IEEE Trans. Patt. Anal. Mach. Intell., 31(3):414-427.

[12]Furukawa, Y., Ponce, J., 2009. Carved visual hulls for image-based modeling. Int. J. Comput. Vis., 81(1):53-67.

[13]Furukawa, Y., Ponce, J., 2010. Accurate, dense, and robust multiview stereopsis. IEEE Trans. Patt. Anal. Mach. Intell., 32(8):1362-1376.

[14]Haro, G., 2012. Shape from silhouette consensus. Patt. Recogn., 45(9):3231-3244.

[15]Hasler, N., Rosenhahn, B., Thormahlen, T., et al., 2009. Markerless motion capture with unsynchronized moving cameras. IEEE Conf. on Computer Vision and Pattern Recognition, p.224-231.

[16]Hauswiesner, S., Khlebnikov, R., Steinberger, M., et al., 2012. Multi-GPU image-based visual hull rendering. 12th Eurographics Symp. on Parallel Graphics and Visualization, p.119-128.

[17]Hofmann, M.H., Davrila, M.G., 2009. Multi-view 3D human pose estimation combining single-frame recovery, temporal integration and model adaptation. IEEE Conf. on Computer Vision and Pattern Recognition, p.2214-2221.

[18]Huang, C.H., Lu, D.M., Diao, C.Y., 2013. Accelerated visual hulls of complex objects using contribution weights. Proc. 7th Int. Conf. on Image and Graphics, p.685-689.

[19]Huang, C.H., Lu, D.M., Diao, C.Y., 2014a. A point cloud representation using plane-space-local-area-color- consistency. J. Comput.-Aided Des. Comput. Graph., 26(8):1297-1303 (in Chinese).

[20]Huang, C.H., Boyer, E., Navab, N., et al., 2014b. Human shape and pose tracking using keyframes. IEEE Conf. on Computer Vision and Pattern Recognition, p.3446-3453.

[21]Kanaujia, A., Haering, N., Taylor, G., et al., 2011. 3D human pose and shape estimation from multi-view imagery. IEEE Computer Vision and Pattern Recognition Workshops, p.49-56.

[22]Kim, D., Dahyot, R., 2012. Bayesian shape from silhouettes. Int. Workshop on Multimedia Understanding Through Semantics, Computation, and Learning, p.78-89.

[23]Laurentini, A., 1994. The visual hull concept for silhouette-based image understanding. IEEE Trans. Patt. Anal. Mach. Intell., 16(2):150-162.

[24]Li, K., Dai, Q.H., Xu, W.L., 2011. Markerless shape and motion capture from multiview video sequences. IEEE Trans. Circ. Syst. Video Technol., 21(3):320-334.

[25]Liu, Y.B., Dai, Q.H., Xu, W.L., 2010. A point-cloud-based multiview stereo algorithm for free-view-point video. IEEE Trans. Vis. Comput. Graph., 16(3):407-418.

[26]Matsuyama, T., Wu, X.J., Takai, T., et al., 2004. Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video. IEEE Trans. Circ. Syst. Video Technol., 14(3):357-369.

[27]Matusik, W., Buehler, C., Raskar, R., et al., 2000. Image-based visual hulls. ACM Special Interest Group on Computer Graphics, p.369-374.

[28]Nakajima, H., Makihara, Y., Hsu, H., et al., 2012. Point cloud transport. 21st Int. Conf. on Pattern Recognition, p.3803-3806.

[29]Nakazawa, M., Mitsugami, I., Makihara, Y., et al., 2012. Dynamic scene reconstruction using asynchronous multiple Kinects. 21st Int. Conf. on Pattern Recognition, p.469-472.

[30]Perez, J.M., Aledo, P.G., Sanchez, P.P., 2012. Real-time voxel-based visual hull reconstruction. Microprocess. Microsyst., 36(5):439-447.

[31]Raeesi N., M.R., Wu, Q.M.J., 2010. A complete visual hull representation using bounding edges. 11th Pacific-Rim Conf. on Multimedia, p.171-182.

[32]Taneja, A., Ballan, L., Pollefeys, M., 2011. Modeling dynamic scenes recorded with freely moving cameras. 10th Asian Conf. on Computer Vision, p.613-626.

[33]Vlasic, D., Peers, P., Baran, I., et al., 2009. Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph., 28(5):174.

[34]Wang, S.Y., Yu, H.M., 2012. Convex relaxation for a 3D spatiotemporal segmentation model using the primal-dual method. J. Zhejiang Univ.-Sci. C (Comput. & Electron), 13(6):428-439.

[35]Wu, X.J., Takizawa, O., Matsuyama, T., 2006. Parallel pipeline volume intersection for real-time 3D shape reconstruction on a PC cluster. IEEE Int. Conf. on Computer Vision Systems, p.1-4.

[36]Xia, D., Li, D.H., Li, Q.G., 2011. A novel approach for computing exact visual hull from silhouettes. Optik, 122(24):2220-2226.

[37]Zhang, Z., Seah, H.S., Quah, C.K., et al., 2011. A multiple camera system with real-time volume reconstruction for articulated skeleton pose tracking. 17th Int. Multimedia Modeling Conf., p.182-192.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE