A New Weight Initialization Method Using Cauchy’s Inequality Based on Sensitivity Analysis

Copyright © 2011 SciRes. JILSA

248

ucts for Second Order Gradient Descent,” Neural Com-

putation, Vol. 14, No. 7, 2002, pp. 1723-1738.

doi:10.1162/08997660260028683

[5] F. Biegler-Konig and F. Barnmann, “A Learning Ago-

rithm for Multilayered Neural Networks Based on Linear

Least Squares Problems,” Neural Networks, Vol. 6, No. 1,

1993, pp. 127-131. doi:10.1016/S0893-6080(05)80077-2

[6] Y. F. Yam and T. W. S. Chow, “Determining Initial

Weights of Feedforward Neural Networks Based on Least

Squares Method,” Neural Processing Letters, Vol. 2, No.

2, 1995, pp. 13-17. doi:10.1007/BF02312350

[7] Y. F. Yam, T. W. S. Chow and C. T. Leung, “A New

Method in Determining the Initial Weights of Feedfor-

ward Neural Networks for Training Enhancement,” Neu-

rocomputing, Vol. 16, No. 1, 1997, pp. 23-32.

doi:10.1016/S0925-2312(96)00058-6

[8] G. P .Drago amd S. Ridella, “Statiscally Controlled Acti-

vation Weight Initialization (SCAWI),” IEEE Transac-

tions on Neural Networks, Vol. 3, No. 4, 1992, pp. 899-

905. doi:10.1109/72.143378

[9] D. Nguyen and B. Widrow, “Improving the Learning

Speed of 2-Layer Neural Networks by Choosing Initial

Values of the Adaptive Weights,” Proceedings of the In-

ternational Joint Conference on Neural Networks, San

Diego, Vol. 3, 17-21 June 1990, pp. 21-26.

doi:10.1109/IJCNN.1990.137819

[10] H. Shimodaira, “A Weight Value Initialization Method

for Improved Learning Performance of the Back Propaga-

tion Algorithm in Neural Networks,” Proceedings of the

sixth Internation Conference on Tools with Artificial In-

telligence, New Orleans, 6-9 November 1994, pp. 672-

675. doi:10.1109/TAI.1994.346429

[11] M. Lehtokangas, J. Saarinen, K. Kaski and P. Huuhtanen,

“Initializing Weights of a Multilayer Perceptron Network

by Using the Orthogonal Least Squares Problem,” Neural

Computation, Vol. 7, No. 5, 1995, pp. 982-999.

doi:10.1162/neco.1995.7.5.982

[12] Y. Liu, C. F. Zhou and Y. W. Chen, “Weight Initializa-

tion of Feedforward Neural Networks by Means of Partial

Least Squares,” International Conference on Maching

Learning and Cybernetics, Dalian, 13-16 August 2006,

pp. 3119-3122.

[13] X. M. Zhang, Y. Q. Chen, N. Ansari and Y. Q. Shi, “Mini-

Max Initialization for Function Approximation,” Neuro-

computing, Vol. 57, 2004, pp. 389-409.

doi:10.1016/j.neucom.2003.10.014

[14] M. Fernandez-Redondo and C. Hernandez-Espinosa, “A Com-

parison among Weight Initialization Methods for Multi-

layer Feedforward Networks,” Proceedings of the IEEE-

INNS-ENNS International Joint Conference on Neural

Networks, Como, Vol. 4, 24-27 July 2000, pp. 543-548 .

[15] T.-C. Hsiao, C.-W. Lin and H. K. Chiang, “Partial Least

Squares Algorithm for Weight Initialization of Backpro-

pagation Network,” Neurocomputing, Vol. 50, 2003, pp.

237-247. doi:10.1016/S0925-2312(01)00708-1

[16] M. Huskan and C. Goerick, “Fast Learning for Problem

Classes Using Knowledge Based Network Initialization,”

Proceedings of International Conference on Neural Net-

works, Como, 24-27 July 2000, pp. 619-624.

[17] D. Erdogmus, O. Fontenla-Romero, J. C. Principe, A. Alon-

so-Betanzos and E. Castillo, “Linear-Leaset-Squares Initia-

lization of Multilayer Perceptrons through Backpropaga-

tion of the Desired Response,” IEEE Transactions of Neu-

ral Networks, Vol. 16, No. 2, 2005, pp. 325-337.

doi:10.1109/TNN.2004.841777

[18] Y. F. Yam and T. W. S. Chow, “A Weight Initialization Me-

thod for Improving Training Speed in Feedforward Neu-

ral Network,” Neurocomputing, Vol. 30, No. 1-4, 2000,

pp. 219-232. doi:10.1016/S0925-2312(99)00127-7

[19] Y. F. Yam and T. W. S. Chow, “Feedforward Networks Trai-

ning Speed Enhancement by Optimal Initialization of the

Synaptic Coefficients,” IEEE Transactions on Neural

Networks, Vol. 12, No. 2, 2001, pp. 430-434.

doi:10.1109/72.914538

[20] E. Castillo , O . Fo nten la-Romer o, A. A. B etanzos and B. Gui-

jarro-Berdinas, “A Global Optimum Approach for One

Layer Neural Networks,” Neural Computation, Vol. 14,

No. 6, 2002, pp. 1429-1449.

doi:10.1162/089976602753713007

[21] E. Castillo, B. Guijarro-Berdinas, O. Fontenla-Romero and

A. A. Betanzos, “A Very Fast Learning Method for Neu-

ral Networks Based on Sensitivity Analysis,” Journal of

Machine Learning Research, Vol. 7, 2006, pp. 1159-1182.

[22] R. A. Fisher, “The Use of Multiple Measurements in Taxo-

nomic Problems,” Annual Eugenics, Vol. 7, No. 2, 1936,

pp. 179-188. doi:10.1111/j.1469-1809.1936.tb02137.x

[23] A. Frank and A. Asuncion, “UCI Machine Learning Re-

pository,” School of Information and Computer Science,

Universty of California, Irvine, 2010.

http://archieve.ics.uci.edu/ml