Full Text:   <3473>

CLC number: TN911.7

On-line Access: 

Received: 2008-02-20

Revision Accepted: 2008-06-19

Crosschecked: 2008-12-26

Cited: 0

Clicked: 5926

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
1. Reference List
Open peer comments

Journal of Zhejiang University SCIENCE A 2009 Vol.10 No.2 P.253-262


A multi-class large margin classifier

Author(s):  Liang TANG, Qi XUAN, Rong XIONG, Tie-jun WU, Jian CHU

Affiliation(s):  Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China

Corresponding email(s):   tang@iipc.zju.edu.cn, rxiong@iipc.zju.edu.cn

Key Words:  Multi-classification, Support vector machine (SVM), Quadratic programming (QP) problem, Large margin

Liang TANG, Qi XUAN, Rong XIONG, Tie-jun WU, Jian CHU. A multi-class large margin classifier[J]. Journal of Zhejiang University Science A, 2009, 10(2): 253-262.

@article{title="A multi-class large margin classifier",
author="Liang TANG, Qi XUAN, Rong XIONG, Tie-jun WU, Jian CHU",
journal="Journal of Zhejiang University Science A",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T A multi-class large margin classifier
%A Liang TANG
%A Tie-jun WU
%A Jian CHU
%J Journal of Zhejiang University SCIENCE A
%V 10
%N 2
%P 253-262
%@ 1673-565X
%D 2009
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.A0820122

T1 - A multi-class large margin classifier
A1 - Liang TANG
A1 - Qi XUAN
A1 - Rong XIONG
A1 - Tie-jun WU
A1 - Jian CHU
J0 - Journal of Zhejiang University Science A
VL - 10
IS - 2
SP - 253
EP - 262
%@ 1673-565X
Y1 - 2009
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.A0820122

Currently there are two approaches for a multi-class support vector classifier (SVC). One is to construct and combine several binary classifiers while the other is to directly consider all classes of data in one optimization formulation. For a K-class problem (K>2), the first approach has to construct at least K classifiers, and the second approach has to solve a much larger optimization problem proportional to K by the algorithms developed so far. In this paper, following the second approach, we present a novel multi-class large margin classifier (MLMC). This new machine can solve K-class problems in one optimization formulation without increasing the size of the quadratic programming (QP) problem proportional to K. This property allows us to construct just one classifier with as few variables in the QP problem as possible to classify multi-class data, and we can gain the advantage of speed from it especially when K is large. Our experiments indicate that MLMC almost works as well as (sometimes better than) many other multi-class SVCs for some benchmark data classification problems, and obtains a reasonable performance in face recognition application on the AR face database.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1] Allwein, E.L., Schapire, R.E., Singer, Y., 2000. Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res., 1:113-141.

[2] Alpaydin, E., Mayoraz, E., 1998. Combining Linear Dichomotizers to Construct Nonlinear Polychotomizers. Technical Report IDIAP-RR, Switzerland.

[3] Angulo, C., Parra, X., Catala, A., 2003. K-SVCR. A support vector machine for multi-class classification. Neurocomputing, 55(1-2):57-77.

[4] Aurenhammer, F., Klein, R., 2000. Voronoi Diagram. In: Sack, J.R., Urrutia, J. (Eds.), Handbook of Computational Geometry. Elsevier Science Publishers, B.V. North-Holland, Amsterdam, p.201-290.

[5] Blake, C.L., Merz, C.J., 1998. UCI Repository of Machine Learning Databases. University of California, Irvine. Http://www.ics.uci.edu/mlearn/MLRepository.html

[6] Bredensteiner, E.J., Bennett, K.P., 1999. Multicategory classification by support vector machines. Comput. Optim. Appl., 12:53-79.

[7] Canu, S., Grandvalet, Y., Guigue, V., Rakotomamonjy, A., 2005. SVM and Kernel Methods Matlab Toolbox. Perception Systèmes et Information, INSA de Rouen, Rouen, France.

[8] Cortes, C., Vapnik, V., 1995. Support vector networks. Machine Learning, 20(3):273-297.

[9] Crammer, K., Singer, Y., 2002. On the learnability and design of output codes for multiclass problems. Machine Learning, 47(2-3):201-233.

[10] Dietterich, T.G., Bakiri, G., 1995. Solving multiclass learning problem via error-correcting output codes. J. Artif. Intell. Res., 2:263-286.

[11] Guermeur, Y., Elisseeff, A., Paugam-Moisy, H., 2000. A New Multi-class SVM Based on a Uniform Convergence Result. Proc. IJCNN, Como, Italy, 4:183-188.

[12] Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R., 2000. A fast iterative nearest point algorithm for support vector machineclassifier design. IEEE Trans. Neural Networks, 11(1):124-136.

[13] Kowalczyk, A., 2000. Maximal Margin Perceptron. In: Smola, A.J., Bartlett, P.L., Scholkopf, B., Schuurmans, D. (Eds.), Advances in Large Margin Classifiers. MIT Press, Cambrige, MA, p.75-113.

[14] Martinez, A.M., Benavente, R., 1998. The AR Face Database. Technical Report, CVC.

[15] Mayoraz, E., Alpaydin, E., 1999. Support Vector Machines for Multi-class Classification. Proc. IWANN, Alicante, Spain, p.833-842.

[16] Moreira, M., Mayoraz, E., 1998. Improved Pairwise Coupling Classification with Correcting Classifiers. Proc. ECML, Chemnitz, Germany, p.160-171.

[17] Platt, J., Cristianini, N., Shawe-Taylor, J., 2000. Large Margin DAGs for Multiclass Classification. In: Solla, S.A., Leen, T.K., Müller, K.R. (Eds.), Advance in Neural Information Process Systems. MIT Press, Cambridge, MA, p.547-553.

[18] Swets, D.L., Weng, J., 1996. Using discriminant eigenfeatures for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell., 18(8):831-836.

[19] Vapnik, V., 2000. The Nature of Statistical Learning Theory. Springer, New York.

[20] Weston, J., Watkins, C., 1998. Multi-class Support Vector Machines. Technical Report, CSD-TR-98-04. Royal Holloway, University of London, Egham, UK.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE