Full Text:   <3186>

CLC number: TP301

On-line Access: 2011-05-09

Received: 2010-10-15

Revision Accepted: 2011-02-23

Crosschecked: 2011-04-07

Cited: 1

Clicked: 7306

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
Open peer comments

Journal of Zhejiang University SCIENCE C 2011 Vol.12 No.5 P.362-370


Integrating outlier filtering in large margin training

Author(s):  Xi-chuan Zhou, Hai-bin Shen, Jie-ping Ye

Affiliation(s):  College of Communication Engineering, Chongqing University, Chongqing 400044, China, School of Electrical Engineering, Zhejiang University, Hangzhou 310027, China, Department of Computer Science and Engineering, Arizona State University, Tempe 85281, USA

Corresponding email(s):   zxc@ccee.cqu.edu.cn

Key Words:  Support vector machines, Outlier filter, Semi-definite programming, Multi-stage relaxation

Xi-chuan Zhou, Hai-bin Shen, Jie-ping Ye. Integrating outlier filtering in large margin training[J]. Journal of Zhejiang University Science C, 2011, 12(5): 362-370.

@article{title="Integrating outlier filtering in large margin training",
author="Xi-chuan Zhou, Hai-bin Shen, Jie-ping Ye",
journal="Journal of Zhejiang University Science C",
publisher="Zhejiang University Press & Springer",

%0 Journal Article
%T Integrating outlier filtering in large margin training
%A Xi-chuan Zhou
%A Hai-bin Shen
%A Jie-ping Ye
%J Journal of Zhejiang University SCIENCE C
%V 12
%N 5
%P 362-370
%@ 1869-1951
%D 2011
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.C1000361

T1 - Integrating outlier filtering in large margin training
A1 - Xi-chuan Zhou
A1 - Hai-bin Shen
A1 - Jie-ping Ye
J0 - Journal of Zhejiang University Science C
VL - 12
IS - 5
SP - 362
EP - 370
%@ 1869-1951
Y1 - 2011
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.C1000361

Large margin classifiers such as support vector machines (SVM) have been applied successfully in various classification tasks. However, their performance may be significantly degraded in the presence of outliers. In this paper, we propose a robust SVM formulation which is shown to be less sensitive to outliers. The key idea is to employ an adaptively weighted hinge loss that explicitly incorporates outlier filtering in the SVM training, thus performing outlier filtering and classification simultaneously. The resulting robust SVM formulation is non-convex. We first relax it into a semi-definite programming which admits a global solution. To improve the efficiency, an iterative approach is developed. We have performed experiments using both synthetic and real-world data. Results show that the performance of the standard SVM degrades rapidly when more outliers are included, while the proposed robust SVM training is more stable in the presence of outliers.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article


[1]Bousquet, O., Elisseeff, A., 2002. Stability and generalization. J. Mach. Learn. Res., 2(3):499-526.

[2]Brodley, C.E., Friedl, M.A., 1996. Identifying and Eliminating Mislabeled Training Instances. Proc. 13th National Conf. on Artificial Intelligence, 1:799-805.

[3]Cortes, C., Vapnik, V., 1995. Support vector networks. Mach. Learn., 20(3):273-297.

[4]Davy, M., Godsill, S., 2002. Detection of Abrupt Spectral Changes Using Support Vector Machines: an Application to Audio Signal Segmentation. Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, p.1313-1316.

[5]Eskin, E., Lee, W., Stolfo, S.J., 2001. Modeling System Calls for Intrusion Detection with Dynamic Window Sizes. Proc. DARPA Information Survivability Conf. and Exposition, p.1-11.

[6]Fawcett, T., Provost, F.J., 1997. Adaptive fraud detection. Data Min. Knowl. Disc., 1(3):291-316.

[7]Frank, A., Asuncion, A., 2010. UCI Machine Learning Repository. School of Information and Computer Science, University of California, Irvine.

[8]Herbrich, R., Weston, J., 2000. Adaptive Margin Support Vector Machines for Classification. Advances in Large Margin Classifiers. MIT Press, Cambridge, Massachusetts, USA, p.281-295.

[9]King, S.P., King, D.M., Astley, K., Tarassenko, L., Hayton, P., Utete, S., 2002. The Use of Novelty Detection Techniques for Monitoring High-Integrity Plant. Proc. Int. Conf. on Control Applications, 1:221-226.

[10]Krause, N., Singer, Y., 2004. Leveraging the Margin More Carefully. Proc. 21st Int. Conf. on Machine Learning, p.1-8.

[11]Laskov, P., Schafer, F., Kotenko, I., 2004. Intrusion Detection in Unlabeled Data with Quarter-Sphere Support Vector Machines. Proc. DIMVA, p.71-82.

[12]Manevitz, L.M., Yousef, M., 2002. One-class SVMs for document classification. J. Mach. Learn. Res., 2(2):139-154.

[13]Ratsch, G., Mika, S., Scholkopf, B., Muller, K.R., 2002. Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE Trans. Pattern Anal. Mach. Intell., 24(9):1184-1199.

[14]Scholkopf, B., Smola, A.J., 2002. Learning with Kernels Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge, Massachusetts, USA, p.135-141.

[15]Song, Q., Hu, W., Xie, W., 2002. Robust support vector machine with bullet hole image classification. IEEE Trans. Syst. Man Cybern. C, 32(4):440-448.

[16]Steinwart, I., Hush, D., Scovel, C., 2005. A classification framework for anomaly detection. J. Mach. Learn. Res., 6:211-232.

[17]Tax, D., Ypma, A., Ypma, E., Duin, R.P.W., 1999. Support Vector Data Description Applied to Machine Vibration Analysis. Annual Conf. of the Advanced School for Computing and Imaging, p.398-405.

[18]Tax, D.M.J., 2001. One-Class Classification: Concept-Learning in the Absence of Counter-Examples. PhD Thesis, Delft University of Technology, Delft, the Netherlands.

[19]Thongkam, J., Xu, G., Zhang, Y., Huang, F., 2008. Support Vector Machine for Outlier Detection in Breast Cancer Survivability Prediction. APWeb Workshop, p.99-109.

[20]Wu, Y., Liu, Y., 2007. Robust truncated hinge loss support vector machines. J. Am. Statist. Assoc., 102(479):974-983.

[21]Xu, L., Crammer, K., Schuurmans, D., 2006. Robust Support Vector Machine Training via Convex Outlier Ablation. Proc. National Conf. of Artificial Intelligence, 21:536-542.

[22]Zhang, T., 2008. Multi-stage Convex Relaxation for Learning with Sparse Regularization. NIPS, p.1929-1936.

Open peer comments: Debate/Discuss/Question/Opinion


Please provide your name, email address and a comment

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2024 Journal of Zhejiang University-SCIENCE