Self-Organized Potential Learning: Enhancing SOM Knowledge to Train Supervised Neural Networks with Improved Interpretation and Generalization Performance

Self-Organized Potential Learning: Enhancing SOM Knowledge to Train Supervised Neural Networks with Improved Interpretation and Generalization Performance

The present paper proposes a new type of learning method called “self-organized potential learning” to improve generalization and interpretation performance. In this method, the self-organizing map (SOM) is used to produce the knowledge (SOM knowledge) on input patterns. SOM knowledge is sometimes redundant and not necessarily effective in training multi-layered neural networks. The present method is introduced to focus on the most important part of the knowledge, which is extracted by considering the potentiality of neurons. For the first approximation, the potentiality is defined in terms of the variance of neurons. Then, neurons with larger potentiality are chosen as the important ones to be used in supervised learning. The method was applied to three problems, namely, artificial data, real second language leaning data and the bio-degeneracy data in the machine learning database. In all cases, it was found that in terms of variance, potentiality was effective in extracting a small number of important input and hidden neurons. Then, generalization performance was greatly improved, in particular when input and hidden neurons’ potentiality were considered with easily interpretable connection weights.