Similar articles

Abstract: An important issue involved in kernel methods is the pre-image problem. However, it is an ill-posed problem, as the solution is usually nonexistent or not unique. In contrast to direct methods aimed at minimizing the distance in feature space, indirect methods aimed at constructing approximate equivalent models have shown outstanding performance. In this paper, an indirect method for solving the pre-image problem is proposed. In the proposed algorithm, an inverse mapping process is constructed based on a novel framework that preserves local linearity. In this framework, a local nonlinear transformation is implicitly conducted by neighborhood subspace scaling transformation to preserve the local linearity between feature space and input space. By extending the inverse mapping process to test samples, we can obtain pre-images in input space. The proposed method is non-iterative, and can be used for any kernel functions. Experimental results based on image denoising using kernel principal component analysis (PCA) show that the proposed method outperforms the state-of-the-art methods for solving the pre-image problem.

基于邻域子空间缩放维持局部线性的原像算法

[1]Abrahamsen, T.J., Hansen, L.K., 2011. Regularized pre-image estimation for kernel PCA de-noising. J. Signal Process. Syst., 65(3):403-412.

[2]Arif, O., Vela, P.A., Daley, W., 2010. Pre-image problem in manifold learning and dimensional reduction methods. Proc. 9th Int. Conf. on Machine Learning and Applications, p.921-924.

[3]Bakir, G.H., Weston, J., Scholkopf, B., 2003. Learning to find pre-images. Adv. Neur. Inform. Process. Syst., 16(7):449-456.

[4]Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J., 2001. From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):643-660.

[5]Gruber, P., Stadlthanner, K., Bohm, M., et al., 2006. Denoising using local projective subspace methods. Neurocomputing, 69(13-15):1485-1501.

[6]Honeine, P., Richard, C., 2011a. A closed-form solution for the pre-image problem in kernel-based machines. J. Signal Process. Syst., 65(3):289-299.

[7]Honeine, P., Richard, C., 2011b. Preimage problem in kernel-based machine learning. IEEE Signal Process. Mag., 28(2):77-88.

[8]Huang, D., Tian, Y.D., de la Torre, F., 2011. Local isomorphism to solve the pre-image problem in kernel methods. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.2761-2768.

[9]Izenman, A.J., 2008. Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. Springer, New York, USA.

[10]Jenssen, R., 2010. Kernel entropy component analysis. IEEE Trans. Pattern Anal. Mach. Intell., 32(5):847-860.

[11]Kallas, M., Honeine, P., Richard, C., et al., 2013. Non-negativity constraints on the pre-image for pattern recognition with kernel machines. Pattern Recog., 46(11):3066-3080.

[12]Kim, K.I., Franz, M.O., Scholkopf, B., 2005. Iterative kernel principal component analysis for image modeling. IEEE Trans. Pattern Anal. Mach. Intell., 27(9):1351-1366.

[13]Kwok, J.T.Y., Tsang, I.W., 2004. The pre-image problem in kernel methods. IEEE Trans. Neur. Networks, 15(6):1517-1525.

[14]Lee, K.C., Ho, J., Kriegman, D., 2005. Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell., 27(5):684-698.

[15]Li, J.W., Su, L., 2008. Combining KPCA and PSO for pattern denoising. Proc. Chinese Conf. on Pattern Recognition, p.1-6.

[16]Li, J.W., Su, L., Cheng, C., 2011. Finding pre-images via evolution strategies. Appl. Soft Comput., 11(6):4183-4194.

[17]Mika, S., Scholkopf, B., Smola, A., et al., 1998. Kernel PCA and de-noising in feature spaces. Adv. Neur. Inform. Process. Syst., 11:536-542.

[18]Nguyen, M.H., de la Torre, F., 2008. Robust kernel principal component analysis. Adv. Neur. Inform. Process. Syst., 21:1185-1192.

[19]Park, J., Kang, D., Kim, J., et al., 2007. SVDD-based pattern denoising. Neur. Comput., 19(7):1919-1938.

[20]Rathi, Y., Dambreville, S., Tannenbaum, A., 2006. Statistical shape analysis using kernel PCA. SPIE, 6064:60641B.

[21]Saul, L.K., Roweis, S.T., 2003. Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learn. Res., 4:119-155.

[22]Scholkopf, B., Smola, A., Muller, K.R., 1997. Kernel principal component analysis. Proc. 7th Int. Conf. on Artificial Neural Networks, p.583-588.

[23]Scholkopf, B., Herbrich, R., Smola, A.J., 2001. A generalized representer theorem. Proc. 14th Annual Conf. on Computational Learning Theory and 5th European Conf. on Computational Learning Theory, p.416-426.

[24]Teixeira, A.R., Tome, A.M., Stadlthanner, K., et al., 2008. KPCA denoising and the pre-image problem revisited. Digit. Signal Process., 18(4):568-580.

[25]Zheng, W.S., Lai, J.H., 2006. Regularized locality preserving learning of pre-image problem in kernel principal component analysis. Proc. 18th Int. Conf. on Pattern Recognition, p.456-459.

[26]Zheng, W.S., Lai, J.H., Yuen, P.C., 2010. Penalized preimage learning in kernel principal component analysis. IEEE Trans. Neur. Networks, 21(4):551-570.