JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering 2020 Vol.21 No.8 P.1191-1205

http://doi.org/10.1631/FITEE.1900156

A non-Lambertian photometric stereo under perspective projection

Author(s): Min Li, Chang-yu Diao, Duan-qing Xu, Wei Xing, Dong-ming Lu
Affiliation(s): College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China; more
Corresponding email(s): liminlim@126.com, dcy@zju.edu.cn
Key Words: Photometric stereo, Three-dimensional reconstruction, Perspective projection, Image decomposition

Share this article to： More <<< Previous Article \|Next Article >>>

Abstract: Under the perspective projection assumption, non-Lambertian photometric stereo is a highly non-linear problem. In this study, we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components, we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

透视投影下的非朗伯光度立体技术

李敏¹，刁常宇²，许端清¹，邢卫¹，鲁东明¹
¹浙江大学计算机科学与技术学院，中国杭州市，310027
²浙江大学艺术与考古学院，中国杭州市，310027

摘要：在透视投影假设下，非朗伯光度立体技术是一个高度非线性问题。本文提出一种透视投影下基于非朗伯反射模型的表面法线和深度重构优化框架。将图像分解为漫反射分量和镜面分量，可同时计算表面法向量和反射率。此外提出一个变分公式，其在表面重构中表现出鲁棒性与有益性。实验结果表明，所提方法能准确重构非朗伯曲面彩色物体表面形状并计算出物体表面的反射率。

关键词：光度立体法；三维重构；透视投影；图像分解

Reference

[1]Agrawal A, Raskar R, Nayar SK, et al., 2005. Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Trans Graph, 24(3):828-835.

[2]Agrawal A, Raskar R, Chellappa R, 2006a. Edge suppression by gradient field transformation using cross-projection tensors. IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.2301-2308.

[3]Agrawal A, Raskar R, Chellappa R, 2006b. What is the range of surface reconstructions from a gradient field? 9^th European Conf on Computer Vision, p.578-591.

[4]Barsky S, Petrou M, 2003. The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans Patt Anal Mach Intell, 25(10):1239-1252.

[5]Chandraker M, Agarwal S, Kriegman D, 2007. Shadowcuts: photometric stereo with shadows. IEEE Conf on Computer Vision and Pattern Recognition, p.1-8.

[6]Chung HS, Jia JY, 2008. Efficient photometric stereo on glossy surfaces with wide specular lobes. IEEE Conf on Computer Vision and Pattern Recognition, p.1-8.

[7]Coleman ENJr, Jain R, 1982. Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Comput Graph Image Process, 18(4):309-328.

[8]Donoho DL, 2006. Compressed sensing. IEEE Trans Inform Theory, 52(4):1289-1306.

[9]Esteban CH, Vogiatzis G, Cipolla R, 2008. Multiview photometric stereo. IEEE Trans Patt Anal Mach Intell, 30(3):548-554.

[10]Finlayson GD, Schaefer G, 2001a. Convex and non-convex illuminant constraints for dichromatic colour constancy. Proc IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.598-604

[11]Finlayson GD, Schaefer G, 2001b. Solving for colour constancy using a constrained dichromatic reflection model. Int J Comput Vis, 42(3):127-144.

[12]Gabay D, Mercier B, 1976. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput Math Appl, 2(1):17-40.

[13]Galliani S, Lasinger K, Schindler K, 2015. Massively parallel multiview stereopsis by surface normal diffusion. Proc IEEE Int Conf on Computer Vision, p.873-881.

[14]Georghiades AS, 2003. Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. Proc 9^th IEEE Int Conf on Computer Vision, Article 816.

[15]Ghosh A, Chen TB, Peers P, et al., 2009. Estimating specular roughness and anisotropy from second order spherical gradient illumination. Comput Graph For, 28(4):1161-1170.

[16]Goldman DB, Curless B, Hertzmann A, et al., 2010. Shape and spatially-varying BRDFs from photometric stereo. IEEE Trans Patt Anal Mach Intell, 32(6):1060-1071.

[17]Guo XJ, Cao XC, Ma Y, 2014. Robust separation of reflection from multiple images. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2187-2194.

[18]Han BJ, Sim JY, 2018. Glass reflection removal using co-saliency-based image alignment and low-rank matrix completion in gradient domain. IEEE Trans Image Process, 27(10):4873-4888.

[19]Han TQ, Shen HL, 2015. Photometric stereo for general BRDFs via reflection sparsity modeling. IEEE Trans Image Process, 24(12):4888-4903.

[20]He XT, Peng YX, 2019. Fine-grained visual-textual representation learning. IEEE Trans Circ Syst Video Technol, online.

[21]He XT, Peng YX, Zhao JJ, 2019a. Fast fine-grained image classification via weakly supervised discriminative localization. IEEE Trans Circ Syst Video Technol, 29(5):1394-1407.

[22]He XT, Peng YX, Zhao JJ, 2019b. Which and how many regions to gaze: focus discriminative regions for fine-grained visual categorization. Int J Comput Vis, 127(9):1235-1255.

[23]Huynh CP, Robles-Kelly A, 2010. A solution of the dichromatic model for multispectral photometric invariance. Int J Comput Vis, 90(1):1-27.

[24]Ikehata S, 2018. CNN-PS: CNN-based photometric stereo for general non-convex surfaces. European Conf on Computer Vision, p.3-19.

[25]Ikehata S, Aizawa K, 2014. Photometric stereo using constrained bivariate regression for general isotropic surfaces. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.2179-2186.

[26]Ikehata S, Wipf D, Matsushita Y, et al., 2012. Robust photometric stereo using sparse regression. IEEE Conf on Computer Vision and Pattern Recognition, p.318-325.

[27]Kong N, Tai YW, Shin SY, 2012. A physically-based approach to reflection separation. IEEE Conf on Computer Vision and Pattern Recognition, p.9-16.

[28]Levin A, Weiss Y, 2007. User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans Patt Anal Mach Intell, 29(9):1647-1654.

[29]Li Y, Brown MS, 2013. Exploiting reflection change for automatic reflection removal. Proc IEEE Int Conf on Computer Vision, p.2432-2439.

[30]Mallick SP, Zickler TE, Kriegman DJ, et al., 2005. Beyond Lambert: reconstructing specular surfaces using color. IEEE Computer Society Conf on Computer Vision and Pattern Recognition, p.619-626.

[31]Nayar SK, Fang XS, Boult T, 1997. Separation of reflection components using color and polarization. Int J Comput Vis, 21(3):163-186.

[32]Papadhimitri T, Favaro P, 2013. A new perspective on uncalibrated photometric stereo. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.1474-1481.

[33]Peng YX, He XT, Zhao JJ, 2018. Object-part attention model for fine-grained image classification. IEEE Trans Image Process, 27(3):1487-1500.

[34]Quéau Y, Wu T, Lauze F, et al., 2017. A non-convex variational approach to photometric stereo under inaccurate lighting. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.350-359.

[35]Quéau Y, Durix B, Wu T, et al., 2018. LED-based photometric stereo: modeling, calibration and numerical solution. J Math Imag Vis, 60(3):313-340.

[36]Santo H, Samejima M, Sugano Y, et al., 2017. Deep photometric stereo network. Proc IEEE Int Conf on Computer Vision, p.501-509.

[37]Schechner YY, Kiryati N, Basri R, 2000. Separation of transparent layers using focus. Int J Comput Vis, 39(1):25-39.

[38]Shafer SA, 1985. Using color to separate reflection components. Color Res Appl, 10(4):210-218.

[39]Shen HL, Han TQ, Li CG, 2017. Efficient photometric stereo using kernel regression. IEEE Trans Image Process, 26(1):439-451.

[40]Shi BX, Tan P, Matsushita Y, et al., 2014. Bi-polynomial modeling of low-frequency reflectances. IEEE Trans Patt Anal Mach Intell, 36(6):1078-1091.

[41]Shi BX, Mo ZP, Wu Z, et al., 2019. A benchmark dataset and evaluation for non-Lambertian and uncalibrated photometric stereo. IEEE Trans Patt Anal Mach Intell, 41(2):271-284.

[42]Shih Y, Krishnan D, Durand F, et al., 2015. Reflection removal using ghosting cues. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.3193-3201.

[43]Solomon F, Ikeuchi K, 1996. Extracting the shape and roughness of specular lobe objects using four light photometric stereo. IEEE Trans Patt Anal Mach Intell, 18(4):449-454.

[44]Sun C, Liu SC, Yang TT, et al., 2016. Automatic reflection removal using gradient intensity and motion cues. Proc 24^th ACM Int Conf on Multimedia, p.466-470. http://doi.org/10.1145/2964284.2967264

[45]Sunkavalli K, Zickler T, Pfister H, 2010. Visibility subspaces: uncalibrated photometric stereo with shadows. 11^th European Conf on Computer Vision, p.251-264.

[46]Tan RT, Ikeuchi K, 2008. Separating reflection components of textured surfaces using a single image. IEEE Trans Patt Anal Mach Intell, 27(2):353-384. https://10.1109/TPAMI.2005.36

[47]Tankus A, Kiryati N, 2005. Photometric stereo under perspective projection. 10^th IEEE Int Conf on Computer Vision, p.611-616.

[48]Wan RJ, Shi BX, Hwee TA, et al., 2016. Depth of field guided reflection removal. IEEE Int Conf on Image Processing, p.21-25.

[49]Wan RJ, Shi BX, Duan LY, et al., 2018. CRRN: multi-scale guided concurrent reflection removal network. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.4777-4785.

[50]Wang F, Ainouz S, Petitjean C, et al., 2017. Specularity removal: a global energy minimization approach based on polarization imaging. Comput Vis Image Underst, 158:31-39.

[51]Woodham RJ, 1979. Photometric stereo: a reflectance map technique for determining surface orientation from image intensity. 22^nd Annual Technical Symp on Image Understanding Systems and Industrial Applications I, p.136-143.

[52]Wu L, Ganesh A, Shi B, et al., 2010. Robust photometric stereo via low-rank matrix completion and recovery. 10^th Asian Conf on Computer Vision, p.703-717.

[53]Yang J, Gong D, Liu LQ, et al., 2018. Seeing deeply and bidirectionally: a deep learning approach for single image reflection removal. Proc European Conf on Computer Vision, p.654-669.

[54]Yang QX, Tang JH, Ahuja N, 2015. Efficient and robust specular highlight removal. IEEE Trans Patt Anal Mach Intell, 37(6):1304-1311.

[55]Yeung SK, Wu TP, Tang CK, et al., 2015. Normal estimation of a transparent object using a video. IEEE Trans Patt Anal Mach Intell, 37(4):890-897.

[56]Yu C, Seo Y, Lee SW, 2010. Photometric stereo from maximum feasible Lambertian reflections. 11^th European Conf on Computer Vision, p.115-126.

[57]Zhang XE, Ng R, Chen QF, 2018. Single image reflection separation with perceptual losses. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.4786-4794.

[58]Zhou ZL, Wu Z, Tan P, 2013. Multi-view photometric stereo with spatially varying isotropic materials. IEEE Conf on Computer Vision and Pattern Recognition, p.1482-1489.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Similar articles

- Go to

透视投影下的非朗伯光度立体技术

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference