JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering 2015 Vol.16 No.10 P.817-828

Beyond bag of latent topics: spatial pyramid matching for scene category recognition

Author(s): Fu-xiang Lu, Jun Huang
Affiliation(s): School of Information Science & Engineering, Lanzhou University, Lanzhou 730000, China; more
Corresponding email(s): lufux@lzu.edu.cn, huangj@sari.ac.cn
Key Words: Scene category recognition, Probabilistic latent semantic analysis, Bag-of-words, Adaptive boosting

Share this article to： More <<< Previous Article \|Next Article >>>

Abstract: We propose a heterogeneous, mid-level feature based method for recognizing natural scene categories. The proposed feature introduces spatial information among the latent topics by means of spatial pyramid, while the latent topics are obtained by using probabilistic latent semantic analysis (pLSA) based on the bag-of-words representation. The proposed feature always performs better than standard pLSA because the performance of pLSA is adversely affected in many cases due to the loss of spatial information. By combining various interest point detectors and local region descriptors used in the bag-of-words model, the proposed feature can make further improvement for diverse scene category recognition tasks. We also propose a two-stage framework for multi-class classification. In the first stage, for each of possible detector/descriptor pairs, adaptive boosting classifiers are employed to select the most discriminative topics and further compute posterior probabilities of an unknown image from those selected topics. The second stage uses the prod-max rule to combine information coming from multiple sources and assigns the unknown image to the scene category with the highest ‘final’ posterior probability. Experimental results on three benchmark scene datasets show that the proposed method exceeds most state-of-the-art methods.

超越隐主题包模型：针对场景类别识别的空间金字塔匹配

目的：随着智能手机、数码相机的普及和互联网的高速发展，基于内容的场景类别识别对于图像数据库标注和检索具有重要意义。在场景类别数目比较多的情况下，本文基于概率隐语义分析（pLSA）和自适应提升（AdaBoost）算法，实现一种鲁棒的场景类别识别算法。
创新点：记录pLSA学习得到的主题的位置关系，提出金字塔主题直方图；在词包（bag-of-words）模型中采用不同的兴趣点提取算子和不同的局部区域描述符，实现异质金字塔单词直方图，显著提升场景识别准确率；提出一种两级多分类算法。
方法：利用期望最大化（EM）算法计算图像或图像块的pLSA主题分布，通过空间金字塔（SP）记录主题之间的大致位置关系；通过对兴趣点提取算子和区域描述符的比较研究，在词包模型中选用稠密兴趣点提取算子和六种区域描述符，从而得到六个金字塔主题直方图用来表示图像；为充分利用各异质金字塔主题直方图信息，先用AdaBoost选取分辨能力强的主题并计算测试图像的后验概率，再由prod-max融合规则确定测试图像的类别。
结论：对于特定的兴趣点提取算子和特定的区域描述符，金字塔主题直方图对所有基准图像库的场景识别率均高于标准pLSA主题直方图；融合各异质金字塔主题直方图显著提高了场景类别识别率。

关键词：场景类别识别；概率隐语义分析；词包；自适应提升

Reference

[1]Freund, Y., Schapire, R.E., 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci., 55(1):119-139.

[2]Harris, C., Stephens, M., 1988. A combined corner and edge detector. Alvey Vision Conf., p.147-151.

[3]Hofmann, T., 1999. Probabilistic latent semantic indexing. Proc. 22nd Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, p.50-57.

[4]Hu, Z.H., Cai, Y.Z., Li, Y.G., et al., 2005. Data fusion for fault diagnosis using multi-class support vector machines. J. Zhejiang Univ.-Sci., 6A(10):1030-1039.

[5]Kadir, T., Brady, M., 2001. Saliency, scale and image description. Int. J. Comput. Vis., 45(2):83-105.

[6]Kwitt, R., Vasconcelos, N., Rasiwasia, N., 2012. Scene recognition on the semantic manifold. European Conf. on Computer Vision, p.359-372.

[7]Lazebnik, S., Schmid, C., Ponce, J., 2006. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.2169-2178.

[8]Li, F.F., Perona, P., 2005. A Bayesian hierarchical model for learning natural scene categories. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.524-531.

[9]Liu, J.G., Shah, M., 2007. Scene modeling using co-clustering. IEEE Int. Conf. on Computer Vision, p.1-7.

[10]Lowe, D.G., 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis., 60(2):91-110.

[11]Lu, F.X., Yang, X.K., Zhang, R., et al., 2009. Image classification based on pyramid histogram of topics. IEEE Int. Conf. on Multimedia and Expo, p.398-401.

[12]Lu, F.X., Yang, X.K., Lin, W.Y., et al., 2011. Image classification with multiple feature channels. Opt. Eng., 50(5):057210.1-057210.9.

[13]Matas, J., Chum, O., Urban, M., et al., 2004. Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput., 22(10):761-767.

[14]Mikolajczyk, K., Schmid, C., 2004. Scale & affine invariant interest point detectors. Int. J. Comput. Vis., 60(1):63-86.

[15]Oliva, A., Torralba, A., 2001. Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis., 42(3):145-175.

[16]Qi, X.B., Xiao, R., Li, C.G., et al., 2014. Pairwise rotation invariant co-occurrence local binary pattern. IEEE Trans. Patt. Anal. Mach. Intell., 36(11):2199-2213.

[17]Quelhas, P., Monay, F., Odobez, J., et al., 2007. A thousand words in a scene. IEEE Trans. Patt. Anal. Mach. Intell., 29(9):1575-1589.

[18]Shechtman, E., Irani, M., 2007. Matching local self-similarities across images and videos. IEEE Conf. on Computer Vision and Pattern Recognition, p.1-8.

[19]Wang, Z.L., Feng, J.S., Yan, S.C., et al., 2013. Linear distance coding for image classification. IEEE Trans. Image Process., 22(2):537-548.

[20]Wu, J.X., 2012. Efficient HIK SVM learning for image classification. IEEE Trans. Image Process., 21(10):4442-4453.

[21]Wu, J.X., Rehg, J.M., 2011. CENTRIST: a visual descriptor for scene categorization. IEEE Trans. Patt. Anal. Mach. Intell., 33(8):1489-1501.

[22]Zhang, J.G., Marszałek, M., Lazebnik, S., et al., 2006. Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis., 73(2):213-238.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Similar articles

- Go to

超越隐主题包模型：针对场景类别识别的空间金字塔匹配

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference