Abstract: Subdivision is a widely used technique for mesh refinement. Classic methods rely on fixed manually defined weighting rules and struggle to generate a finer mesh with appropriate details, while advanced neural subdivision methods achieve data-driven nonlinear subdivision but lack robustness, suffering from limited subdivision levels and artifacts on novel shapes. To address these issues, this paper introduces a neural mesh refinement (NMR) method that uses the geometric structural priors learned from fine meshes to adaptively refine coarse meshes through subdivision, demonstrating robust generalization. Our key insight is that it is necessary to disentangle the network from non-structural information such as scale, rotation, and translation, enabling the network to focus on learning and applying the structural priors of local patches for adaptive refinement. For this purpose, we introduce an intrinsic structure descriptor and a locally adaptive neural filter. The intrinsic structure descriptor excludes the non-structural information to align local patches, thereby stabilizing the input feature space and enabling the network to robustly extract structural priors. The proposed neural filter, using a graph attention mechanism, extracts local structural features and adapts learned priors to local patches. Additionally, we observe that Charbonnier loss can alleviate over-smoothing compared to L2 loss. By combining these design choices, our method gains robust geometric learning and locally adaptive capabilities, enhancing generalization to various situations such as unseen shapes and arbitrary refinement levels. We evaluate our method on a diverse set of complex three-dimensional (3D) shapes, and experimental results show that it outperforms existing subdivision methods in terms of geometry quality. See https://zhuzhiwei99.github.io/NeuralMeshRefinement for the project page.

神经网格细化

[1]Abutbul A, Elidan G, Katzir L, et al., 2020. DNF-Net: a neural architecture for tabular data.

[2]Atwood J, Towsley D, 2016. Diffusion-convolutional neural networks. Proc 30^th Int Conf on Neural Information Processing Systems, p.2001-2009.

[3]Beatty JC, Dyn N, Levine D, et al., 1990. A butterfly subdivision scheme for surface interpolation with tension control. ACM Trans Graph, 9(2):160-169.

[4]Bronstein AM, Bronstein MM, Kimmel R, 2009. Numerical Geometry of Non-rigid Shapes. Springer, New York, USA.

[5]Catmull E, Clark J, 1998. Recursively generated B-spline surfaces on arbitrary topological meshes. Proc Seminal Graphics: Pioneering Efforts That Shaped the Field, p.183-188.

[6]Charbonnier P, Blanc-Féraud L, Aubert G, et al., 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. Proc 1^st Int Conf on Image Processing, p.168-172.

[7]Chen HG, He XH, Qing LB, et al., 2022. Real-world single image super-resolution: a brief review. Inform Fus, 79:124-145.

[8]Chen Y, Zhao JY, Shi CW, et al., 2020. Mesh convolution: a novel feature extraction method for 3D nonrigid object classification. IEEE Trans Multim, 23:3098-3111.

[9]Chen YC, Kim V, Aigerman N, et al., 2023. Neural progressive meshes. Proc ACM SIGGRAPH, Article 84.

[10]Choi Y, Jeong JB, Lee S, et al., 2022. Overview of the video-based dynamic mesh coding (V-DMC) standard work. Proc 13^th Int Conf on Information and Communication Technology Convergence, p.578-581.

[11]Dai A, Nießner M, 2019. Scan2Mesh: from unstructured range scans to 3D meshes. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5569-5578.

[12]Dai T, Cai JR, Zhang YB, et al., 2019. Second-order attention network for single image super-resolution. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.11057-11066.

[13]DeRose T, Kass M, Truong T, 1998. Subdivision surfaces in character animation. Proc 25^th Annual Conf on Computer Graphics and Interactive Techniques, p.85-94.

[14]Dong C, Loy CC, Tang XO, 2016a. Accelerating the super-resolution convolutional neural network. Proc 14^th European Conf on Computer Vision, p.391-407.

[15]Dong C, Loy CC, He KM, et al., 2016b. Image super-resolution using deep convolutional networks. IEEE Trans Patt Anal Mach Intell, 38(2):295-307.

[16]Doo D, Sabin M, 1998. Behaviour of recursive division surfaces near extraordinary points. Proc Seminal Graphics: Pioneering Efforts That Shaped the Field, p.177-181.

[17]Dyn N, 2006. Three families of nonlinear subdivision schemes. Stud Comput Math, 12:23-38.

[18]Feng YT, Feng YF, You HX, et al., 2019. MeshNet: mesh neural network for 3D shape representation. Proc 33^rd AAAI Conf on Artificial Intelligence, p.8279-8286.

[19]Garland M, Heckbert PS, 1997. Surface simplification using quadric error metrics. Proc 24^th Annual Conf on Computer Graphics and Interactive Techniques, p.209-216.

[20]Haim N, Segol N, Ben-Hamu H, et al., 2019. Surface networks via general covers. Proc IEEE/CVF Int Conf on Computer Vision, p.632-641.

[21]Han B, Zhang XY, Ren S, 2022. PU-GACNet: graph attention convolution network for point cloud upsampling. Image Vis Comput, 118:104371.

[22]Hanocka R, Hertz A, Fish N, et al., 2019. MeshCNN: a network with an edge. ACM Trans Graph, 38(4):90.

[23]Haris M, Shakhnarovich G, Ukita N, 2018. Deep back-projection networks for super-resolution. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.1664-1673.

[24]Hertz A, Hanocka R, Giryes R, et al., 2020. Deep geometric texture synthesis. ACM Trans Graph, 39(4):108.

[25]Hoppe H, DeRose T, Duchamp T, et al., 1994. Piecewise smooth surface reconstruction. Proc 21^st Annual Conf on Computer Graphics and Interactive Techniques, p.295-302.

[26]Hu SM, Liu ZN, Guo MH, et al., 2022. Subdivision-based mesh convolution networks. ACM Trans Graph, 41(3):25.

[27]Hu YX, Schneider T, Wang BL, et al., 2020. Fast tetrahedral meshing in the wild. ACM Trans Graph, 39(4):117.

[28]Huang JW, Zhang HT, Yi L, et al., 2019. TextureNet: consistent local parametrizations for learning from high-resolution signals on meshes. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4435-4444.

[29]Jiao JB, Tu WC, He SF, et al., 2017. FormResNet: formatted residual learning for image restoration. Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops, p.1034-1042.

[30]Karčiauskas K, Peters J, 2007. Concentric tessellation maps and curvature continuous guided surfaces. Comput Aided Geom Des, 24(2):99-111.

[31]Kim D, Shin M, Paik J, 2023. PU-Edgeformer: edge transformer for dense prediction in point cloud upsampling. Proc IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1-5.

[32]Kim J, Lee JK, Lee KM, 2016. Deeply-recursive convolutional network for image super-resolution. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1637-1645.

[33]Kingma DP, Ba J, 2015. Adam: a method for stochastic optimization. Proc 3^rd Int Conf on Learning Representations.

[34]Lahav A, Tal A, 2020. MeshWalker: deep mesh understanding by random walks. ACM Trans Graph, 39(6):263.

[35]Lai WS, Huang JB, Ahuja N, et al., 2017. Deep Laplacian pyramid networks for fast and accurate super-resolution. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.5835-5843.

[36]Levie R, Monti F, Bresson X, et al., 2019. CayleyNets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process, 67(1):97-109.

[37]Levin A, 2006. Modified subdivision surfaces with continuous curvature. ACM Trans Graph, 25(3):1035-1040.

[38]Li RH, Li XZ, Fu CW, et al., 2019. PU-GAN: a point cloud upsampling adversarial network. Proc IEEE/CVF Int Conf on Computer Vision, p.7202-7211.

[39]Lim B, Son S, Kim H, et al., 2017. Enhanced deep residual networks for single image super-resolution. Proc IEEE Conf on Computer Vision and Pattern Recognition Workshops, p.1132-1140.

[40]Lim I, Dielen A, Campen M, et al., 2019. A simple approach to intrinsic correspondence learning on unstructured 3D meshes. Proc European Conf on Computer Vision, p.349-362.

[41]Liu HTD, Kim VG, Chaudhuri S, et al., 2020. Neural subdivision. ACM Trans Graph, 39(4):124.

[42]Liu K, Ma N, Wang ZH, et al., 2023. Implicit neural distance optimization for mesh neural subdivision. Proc IEEE Int Conf on Multimedia and Expo, p.2039-2044.

[43]Loop C, 1987. Smooth Subdivision Surfaces Based on Triangles. MS Thesis, University of Utah, Salt Lake City, USA.

[44]Maron H, Galun M, Aigerman N, et al., 2017. Convolutional neural networks on surfaces via seamless toric covers. ACM Trans Graph, 36(4):71.

[45]Milano F, Loquercio A, Rosinol A, et al., 2020. Primal-dual mesh convolutional neural networks. Proc 34^th Int Conf on Neural Information Processing Systems, Article 81.

[46]Monti F, Shchur O, Bojchevski A, et al., 2018. Dual-primal graph convolutional networks.

[47]Nair V, Hinton GE, 2010. Rectified linear units improve restricted Boltzmann machines. Proc 27^th Int Conf on Machine Learning, p.807-814.

[48]Oliensis J, 1993. Local reproducible smoothing without shrinkage. IEEE Trans Patt Anal Mach Intell, 15(3):307-312.

[49]Paszke A, Gross S, Massa F, et al., 2019. PyTorch: an imperative style, high-performance deep learning library. Proc 33^rd Int Conf on Neural Information Processing Systems, Article 721.

[50]Perez D, Shen YZ, Li J, 2023. Mesh convolutional networks with face and vertex feature operators. IEEE Trans Vis Comput Graph, 29(3):1678-1690.

[51]Poulenard A, Ovsjanikov M, 2018. Multi-directional geodesic neural networks via equivariant convolution. ACM Trans Graph, 37(6):236.

[52]Qi CR, Su H, Mo KC, et al., 2017. PointNet: deep learning on point sets for 3D classification and segmentation. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.77-85.

[53]Qian GC, Abualshour A, Li GH, et al., 2021. PU-GCN: point cloud upsampling using graph convolutional networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.11678-11687.

[54]Schaefer S, Vouga E, Goldman R, 2008. Nonlinear subdivision through nonlinear averaging. Comput Aided Geom Des, 25(3):162-180.

[55]Sharp N, Attaiki S, Crane K, et al., 2022. DiffusionNet: discretization agnostic learning on surfaces. ACM Trans Graph, 41(3):27.

[56]Smirnov D, Solomon J, 2021. HodgeNet: learning spectral geometry on triangle meshes. ACM Trans Graph, 40(4):166.

[57]Sorkine O, Cohen-Or D, Lipman Y, et al., 2004. Laplacian surface editing. Proc Eurographics/ACM SIGGRAPH Symp on Geometry Processing, p.175-184.

[58]Stam J, 1998. Exact evaluation of Catmull-Clark subdivision surfaces at arbitrary parameter values. Proc 25^th Annual Conf on Computer Graphics and Interactive Techniques, p.395-404.

[59]Tai Y, Yang J, Liu XM, 2017. Image super-resolution via deep recursive residual network. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2790-2798.

[60]Tatarchenko M, Park J, Koltun V, et al., 2018. Tangent convolutions for dense prediction in 3D. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.3887-3896.

[61]Taubin G, 1995. A signal processing approach to fair surface design. Proc 22^nd Annual Conf on Computer Graphics and Interactive Techniques, p.351-358.

[62]Vaxman A, Müller C, Weber O, 2018. Canonical Möbius subdivision. ACM Trans Graph, 37(6):227.

[63]Veličković P, Cucurull G, Casanova A, et al., 2018. Graph attention networks.

[64]Wang NY, Zhang YD, Li ZW, et al., 2018. Pixel2Mesh: generating 3D mesh models from single RGB images. Proc 15^th European Conf on Computer Vision, p.55-71.

[65]Wang XT, Yu K, Wu SX, et al., 2018. ESRGAN: enhanced super-resolution generative adversarial networks. Proc European Conf on Computer Vision, p.63-79.

[66]Wang Y, Sun YB, Liu ZW, et al., 2019. Dynamic graph CNN for learning on point clouds. ACM Trans Graph, 38(5):146.

[67]Wang YF, Perazzi F, McWilliams B, et al., 2018. A fully progressive approach to single-image super-resolution. IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops.

[68]Wang YF, Wu SH, Huang H, et al., 2019. Patch-based progressive 3D point set upsampling. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.5951-5960.

[69]Wang ZH, Chen J, Hoi SCH, 2021. Deep learning for image super-resolution: a survey. IEEE Trans Patt Anal Mach Intell, 43(10):3365-3387.

[70]Yang WM, Zhang XC, Tian YP, et al., 2019. Deep learning for single image super-resolution: a brief review. IEEE Trans Multim, 21(12):3106-3121.

[71]Yu LQ, Li XZ, Fu CW, et al., 2018. PU-Net: point cloud upsampling network. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2790-2799.

[72]Zhou QN, Jacobson A, 2016. Thingi10K: a dataset of 10,000 3D-printing models.

[73]Zorin D, 2007. Subdivision on arbitrary meshes: algorithms and theory. Proc Mathematics and Computation in Imaging Science and Information Processing, p.1-46.

[74]Zorin D, Schröder P, Sweldens W, 1996. Interpolating subdivision for meshes with arbitrary topology. Proc 23^rd Annual Conf on Computer Graphics and Interactive Techniques, p.189-192.