Similar articles

Abstract: As the popularity of digital images is rapidly increasing on the Internet, research on technologies for semantic image classification has become an important research topic. However, the well-known content-based image classification methods do not overcome the so-called semantic gap problem in which low-level visual features cannot represent the high-level semantic content of images. image classification using visual and textual information often performs poorly since the extracted textual features are often too limited to accurately represent the images. In this paper, we propose a semantic image classification approach using multi-context analysis. For a given image, we model the relevant textual information as its multi-modal context, and regard the related images connected by hyperlinks as its link context. Two kinds of context analysis models, i.e., cross-modal correlation analysis and link-based correlation model, are used to capture the correlation among different modals of features and the topical dependency among images induced by the link structure. We propose a new collective classification model called relational support vector classifier (RSVC) based on the well-known Support Vector Machines (SVMs) and the link-based correlation model. Experiments showed that the proposed approach significantly improved classification accuracy over that of SVM classifiers using visual and/or textual features.

[1] Cai, D., Yu, S., Wen, J.R., Ma, W.Y., 2003. VIPS: A Vision-base Page Segmentation Algorithm. Technical Report, MSR-TR-2003-79, Microsoft Research Asia.

[2] Cai, D., He, X.F., Li, Z.W., Ma, W.Y., Wen, J.R., 2004. Hierarchical Clustering of WWW Image Search Results Using Visual, Textual and Link Analysis. Proceedings of 12th ACM International Conference on Multimedia, New York, USA, p.952-959.

[3] Chen, Z., Liu, W.Y., Zhang, F., Li, M.J., Zhang, H.J., 2001. Web mining for Web image retrieval. Journal of American Society of Infomation Science and Technology, 52(10):831-839.

[4] Cristianini, N., Shawe-Talyor, J., Lodhi, H., 2002. Latent semantic kernels. Journal of Intelligent Information Systems, 18(2/3):127-152.

[5] Deerwester, S., Dumais, S.T., Furnas, G.W., Landauer, T.K., Harshman, R., 1990. Indexing by latent semantic analysis. Journal of American Society of Infomation Science and Technology, 41(6):389-401.

[6] Gärtner, T., 2003. A survey of kernels for structured data. SIGKDD Explorations, 5(1):49-58.

[7] Jensen, D., Neville, J., Gallagher, B., 2004. Why Collective Inference Improves Relational Classification. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA, p.593-598.

[8] Joachims, T., Cristianini, N., Shawe-Talyor, J., 2001. Composite Kernels for Hypertext Categorization. Proceedings of the International Conference on Machine Learning (ICML-2001), Morgan Kaunfmann Publishers, San Francisco, US. p.250-257.

[9] Kandola, J., Shawe-Talyor, J., Cristianini, N., 2002. Learning Semantic Similarity. Proceedings of International Conference on Advances in Information Processing System (NIPS).

[10] Kondor, R.I., Lafferty, J., 2002. Diffusion Kernels on Graphs and Other Discrete Structures. Proceedings of the 19th International Conference on Machine Learning (ICML02), p.315-322.

[11] Lempel, R., Soffer, A., 2002. PicASHOW: pictorial authority search by hyperlinks on the Web. ACM Transactions on Information Systems, 20(1):1-24.

[12] Li, D.G., Dimitrova, N., Li, M.K., Sethi, I.K., 2003. Multimedia Content Processing through Cross-modal Association. Proceedings of 11th ACM International Conference on Multimedia, Berkeley, California, USA, p.604-611.

[13] Ma, W.Y., Zhang, H.J., 1998. Content-based Image Indexing and Retrieval. In: Furht, B. (Ed.), Handbook of Multimedia Computing. CRC Press, Boca Raton, FL.

[14] Mohr, J., Obermayer, K., 2005. A Topographic Support Vector Machine: Classification Using Local Label Configurations. Advances in Neural Information Processing Systems 17, MIT Press, Cambridge, MA, p.929-936.

[15] Neville, J., Jensen, D., 2003. Collective Classification with Relational Dependency Networks. Proceedings of 2nd Multi-Relational Data Mining Workshop, 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, p.77-91.

[16] Paek, S., Sable, C.L., Hatzivassiloglou, V., Jaimes, A., Schiffman, B.H., Chang, S.F., McKeown, K.R., 1999. Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs. Proceedings of ACM SIGIR’99, Workshop on Multimedia Indexing and Retrieval.

[17] Schölkopf, B., 2000. Statistical Learning and Kernel Methods. Technical Report, MSR-TR-2000-23, Microsoft Research.

[18] Siolas, G., d’Alché-Buc, F., 2000. Support Machine Learning Based on Semantic Kernel for Text Categorization. Proceedings of the International Joint Conference on Neural Network (IJCNN).

[19] Tian, Y.H., Huang, T.J., Gao, W., 2004. Two-phase Web site classification based on Hidden Markov Tree models. International Journal: Web Intelligence and Agent System, 4(2):249-264.

[20] Tian, Y.H., Huang, T.J., Gao, W., 2005. Latent linkage semantic kernels for collective classification of link data. Journal of Intelligent Information Systems (in Press).

[21] Vailaya, A., Figueiredo, M., A.T., Jain, A.K., Zhang, H.J., 2001. Image classification for content-based indexing. IEEE Trans. Image Processing, 10(1):117-129.

[22] Wang, X.J., Ma, W.Y., Xue, G.R., Li, X., 2004. Multi-Model Similarity Propagation and its Application for Web Image Retrieval. Proceedings of 12th ACM International Conference on Multimedia, New York, USA, p.944-951.

[23] Yang, Y., Slattery, S., Ghani, R., 2002. A study of approaches to hypertext categorization. Journal of Intelligent Information System, 18(2/3):219-241.

[24] Yu, H., Li, M., Zhang, H.J., Feng, J., 2002. Color Exture Moments for Content-based Image Retrieval. International Conference on Image Processing, p.28.

[25] Zhao, R., Grosky, W.I., 2002. Narrowing the emantic gap—improved text-based Web document retrieval using visual features. IEEE Trans. Multimedia, 4(2):189-200.