Auto-associative memory with two-stage dynamics of non-monotonic neurons

Hiro-Fumi Yanai and Shun-ichi Amari


abstract

Dynamical properties of a neural auto-associative memory with two-stage neurons are investigated theoretically. The two-stage neuron is a model whose output is determined by a two-stage nonlinear function of the internal field of the neuron (internal field is a weighted sum of outputs of the other neurons). The model is general, including non-monotonic neurons as well as monotonic ones. Recent studies on associative memory revealed superiority of non-monotonic neurons to monotonic ones. The present paper supplies theoretical verification on the high performance of non-monotonic neurons, and proves that the capacity of the auto-associative memory with two-stage neurons is O(n/sqrt{log n}), in contrast to O(n/log n) of simple threshold neurons . There is also a discussion of recall processes, where the radius of basin of attraction of memorized patterns is clarified. Intuitive explanation on why the performance is improved by non-monotonic neurons is also provided by showing the correspondence of the recall processes of the two-stage-neuron net and orthogonal learning.


Download [ PS file (*.ps.Z) (165KB; 22 pages)]


Back to Publications