Similar articles

Abstract: Human information processing depends mainly on billions of neurons which constitute a complex neural network, and the information is transmitted in the form of neural spikes. In this paper, we propose a spiking neural network (SNN), named MD-SNN, with three key features: (1) using receptive field to encode spike trains from images; (2) randomly selecting partial spikes as inputs for each neuron to approach the absolute refractory period of the neuron; (3) using groups of neurons to make decisions. We test MD-SNN on the MNIST data set of handwritten digits, and results demonstrate that: (1) Different sizes of receptive fields influence classification results significantly. (2) Considering the neuronal refractory period in the SNN model, increasing the number of neurons in the learning layer could greatly reduce the training time, effectively reduce the probability of over-fitting, and improve the accuracy by 8.77%. (3) Compared with other SNN methods, MD-SNN achieves a better classification; compared with the convolution neural network, MD-SNN maintains flip and rotation invariance (the accuracy can remain at 90.44% on the test set), and it is more suitable for small sample learning (the accuracy can reach 80.15% for 1000 training samples, which is 7.8 times that of CNN).

基于感受野编码的多神经元决策脉冲神经网络

[1]Al-Amri SS, Kalyankar NV, Khamitkar SD, 2010. Image segmentation by using edge detection. Int J Comput Sci Eng, 2(3):804-807.

[2]Berry MJII, Meister M, 1998. Refractoriness and neural precision. Proc Conf on Advances in Neural Information Processing Systems 10, p.110-116.

[3]Bi GQ, Poo MM, 2001. Synaptic modification by correlated activity: Hebb’s postulate revisited. Ann Rev Neurosci, 24(1):139-166.

[4]Brette R, Gerstner W, 2005. Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J Neurophysiol, 94(5):3637-3642.

[5]Burt P, Adelson E, 1983. The laplacian pyramid as a compact image code. IEEE Trans Commun, 31(4):532-540.

[6]Canny J, 1986. A computational approach to edge detection. IEEE Trans Patt Anal Mach Intell, 8(6):679-698.

[7]Coates A, Ng A, Lee H, 2011. An analysis of single-layer networks in unsupervised feature learning. Proc 14th Int Conf on Artificial Intelligence and Statistics, p.215-223.

[8]Dasgupta S, Stevens CF, Navlakha S, 2017. A neural algorithm for a fundamental computing problem. Science, 358(6364):793-796.

[9]Dora S, Suresh S, Sundararajan N, 2015a. A sequential learning algorithm for a spiking neural classifier. Appl Soft Comput, 36:255-268.

[10]Dora S, Sundaram S, Sundararajan N, 2015b. A two stage learning algorithm for a growing-pruning spiking neural network for pattern classification problems. Int Joint Conf on Neural Networks, p.1-7.

[11]Dora S, Subramanian K, Suresh S, et al., 2016. Development of a self-regulating evolving spiking neural network for classification problem. Neurocomputing, 171:1216-1229.

[12]Dora S, Suresh S, Sundararajan N, 2017. Online meta-neuron based learning algorithm for a spiking neural classifier. Inform Sci, 414:19-32.

[13]Fukushima K, 1980. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern, 36(4):193-202.

[14]Gerstner W, Kistler W, 2002. Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge, UK.

[15]Ghosh Dastidar S, Adeli H, 2007. Improved spiking neural networks for eeg classification and epilepsy and seizure detection. Integr Comput Aided Eng, 14(3):187-212.

[16]Gilbert CD, Wiesel TN, 1992. Receptive field dynamics in adult primary visual cortex. Nature, 356(6365):150-152.

[17]Gütig R, Sompolinsky H, 2006. The tempotron: a neuron that learns spike timing-based decisions. Nat Neurosci, 9(3):420-428.

[18]Hannun AY, Case C, Casper J, et al., 2014. Deep speech: Scaling up end-to-end speech recognition. https://arxiv.org/abs/1412.5567

[19]He KM, Zhang XY, Ren SQ, et al., 2016. Deep residual learning for image recognition. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.770-778.

[20]Hodgkin AL, Huxley AF, 1952. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol, 117(4):500-544.

[21]Hubel DH, Wiesel TN, 1962. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol, 160(1):106-154.

[22]Hussain S, Liu SC, Basu A, 2014. Improved margin multi-class classification using dendritic neurons with morphological learning. IEEE Int Symp on Circuits and Systems, p.2640-2643.

[23]Izhikevich EM, 2001. Resonate-and-fire neurons. Neur Networks, 14(6-7):883-894.

[24]Izhikevich EM, 2003. Simple model of spiking neurons. IEEE Trans Neur Networks, 14(6):1569-1572.

[25]Izhikevich EM, 2004. Which model to use for cortical spiking neurons? IEEE Trans Neur Networks, 15(5):1063-1070.

[26]LeCun Y, Bengio Y, Hinton G, 2015. Deep learning. Nature, 521(7553):436-444.

[27]Legenstein R, Naeger C, Maass W, 2006. What can a neuron learn with spike-timing-dependent plasticity? Neur Comput, 17(11):2337-2382.

[28]Ma YQ, Wu H, Zhu MJ, et al., 2017. Reconstruction of visual image from functional magnetic resonance imaging using spiking neuron model. IEEE Trans Cogn Dev Syst, in press.

[29]Maass W, 1997. Networks of spiking neurons: the third generation of neural network models. Neur Networks, 10(9):1659-1671.

[30]Masquelier T, Guyonneau R, Thorpe SJ, 2009. Competitive stdp-based spike pattern learning. Neur Comput, 21(5):1259-1276.

[31]Merolla P, Arthur J, Akopyan F, et al., 2011. A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm. IEEE Custom Integrated Circuits Conf, p.1-4.

[32]Ponulak F, Kasiński A, 2010. Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting. Neur Comput, 22(2):467-510.

[33]Rodieck RW, 1965. Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vis Res, 5(11):583-601.

[34]Schmidhuber J, 2015. Deep learning in neural networks: An overview. Neur networks, 61:85-117.

[35]Sobel I, 2014. History and definition of the sobel operator. https://www.scribd.com/document/271811982/History-and-Definition-of-Sobel-Operator

[36]Tang H, Yu Q, Tan KC, 2012. Learning real-world stimuli by single-spike coding and tempotron rule. Int Joint Conf on Neural Networks, p.1-6.

[37]Tavanaei A, Maida AS, 2015. A minimal spiking neural network to rapidly train and classify handwritten digits in binary and 10-digit tasks. Int J Adv Res Artif Intell, 4(7):1-8.

[38]Thorpe S, Delorme A, van Rullen R, 2001. Spike-based strategies for rapid processing. Neur Netw, 14(67):715- 725.

[39]Victor JD, Purpura KP, 1996. Nature and precision of temporal coding in visual cortex: a metric-space analysis. J Neurophysiol, 76(2):1310-1326.

[40]Wade JJ, Mcdaid LJ, Santos JA, et al., 2010. SWAT: a spiking neural network training algorithm for classification problems. IEEE Trans Neur Networks, 21(11):1817-1830.

[41]Xie XR, Qu H, Yi Z, et al., 2017. Efficient training of supervised spiking neural network via accurate synaptic-efficiency adjustment method. IEEE Trans Neur Networks, 28(6):1411-1424.

[42]Yeomans JS, 1979. The absolute refractory periods of self-stimulation neurons. Phys Behav, 22(5):911-919.

[43]Yu Q, Tang HJ, Tan KC, et al., 2013. Rapid feedforward computation by temporal encoding and learning with spiking neurons. IEEE Trans Neur Networks, 24(10):1539-1552.

[44]Yu Q, Tang HJ, Tan KC, et al., 2014. A brain-inspired spiking neural network model with temporal encoding and learning. Neurocomputing, 138:3-13.

[45]Zenke F, Ganguli S, 2017. Superspike: supervised learning in multi-layer spiking neural networks. https://arxiv.org/abs/1705.11146