Publishing Service

Polishing & Checking

Journal of Zhejiang University SCIENCE C

ISSN 1869-1951(Print), 1869-196x(Online), Monthly

A pruning algorithm with L1/2 regularizer for extreme learning machine

Abstract: Compared with traditional learning methods such as the back propagation (BP) method, extreme learning machine provides much faster learning speed and needs less human intervention, and thus has been widely used. In this paper we combine the L1/2 regularization method with extreme learning machine to prune extreme learning machine. A variable learning coefficient is employed to prevent too large a learning increment. A numerical experiment demonstrates that a network pruned by L1/2 regularization has fewer hidden nodes but provides better performance than both the original network and the network pruned by L2 regularization.

Key words: Extreme learning machine (ELM), L1/2 regularizer, Network pruning

Chinese Summary  <45> 利用L1/2正则化进行极端学习机的网络修剪算法

研究背景:1. 神经网络有着广泛的应用,但收敛速度慢、精度低,影响了它的发展。相较于传统的神经网络,极端学习机克服了这些缺点,它不仅提供更快的学习速度,而且只需较少的人工干预,这些优点使得极端学习机得到了广泛应用。2. 相比于L1和L2正则化,L1/2正则化的解具有更好的稀疏性;而与L0正则化相比,它又更容易求解。
创新要点:将L1/2正则化方法与极端学习机结合,利用L1/2正则化较好的稀疏性,修剪极端学习机的网络结构。
方法提亮:极小化的目标函数中含有L1/2范数,当权值变得较小时,其导数值会较大。为了阻止权值过快增长,提出一个可变学习率。
重要结论:数据实验表明,相比于原始的极端学习机算法和带L2正则化的极端学习机算法,带L1/2正则化的极端学习机算法不仅拥有较少隐节点,并且拥有更好泛化能力。

关键词组:极端学习机,L1/2正则化,网络修剪


Share this article to: More

Go to Contents

References:

<Show All>

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





DOI:

10.1631/jzus.C1300197

CLC number:

TP312

Download Full Text:

Click Here

Downloaded:

3535

Download summary:

<Click Here> 

Downloaded:

2201

Clicked:

8007

Cited:

3

On-line Access:

2014-01-29

Received:

2013-07-22

Revision Accepted:

2013-10-14

Crosschecked:

2014-01-15

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952276; Fax: +86-571-87952331; E-mail: jzus@zju.edu.cn
Copyright © 2000~ Journal of Zhejiang University-SCIENCE