This paper proposes an associative memory model based on a coupled system of Gaussian maps. A one-dimensional Gaussian map describes a discrete-time dynamical system, and the coupled system of Gaussian maps can generate various phenomena including asymmetric fixed and periodic points. The Gaussian associative memory can effectively recall one of the stored patterns, which were triggered by an input pattern by associating the asymmetric two-periodic points observed in the coupled system with the binary values of output patterns. To investigate the Gaussian associative memory model, we formed its reduced model and analyzed the bifurcation structure. Pseudo-patterns were observed for the proposed model along with other conventional associative memory models, and the obtained patterns were related to the high-order or quasi-periodic points and the chaotic trajectories. In this paper, the structure of the Gaussian associative memory and its reduced models are introduced as well as the results of the bifurcation analysis are presented. Furthermore, the output sequences obtained from simulation of the recalling process are presented. We discuss the mechanism and the characteristics of the Gaussian associative memory based on the results of the analysis and the simulations conducted.
Previous studies have proposed multiple-associative memory models based on dynamical systems [
Hopfield model has been extended to develop bidirectional and multidirectional associative memories [
The retrieval ability and the storage capacity are important points to be considered when evaluating associative memory models and have been the subject of many studies. One major problem associated with the associative memory model is the generation of the pseudo-patterns. How to avoid the generation of the pseudo-patterns has received remarkable attention in associative memory research community.
However, the mechanisms that are used to effectively recall the stored patterns and to generate pseudo-patterns have not been well elucidated. Herein, we propose a novel associative memory model comprised of Gaussian map to investigate the mechanism of the associative memory based on the qualitative bifurcation theory.
The Gaussian map is a one-dimensional dynamical system which generates various phenomena including periodic points and chaos [
This study investigates the bifurcation structure of periodic points observed in the high dimensional coupled network of the Gaussian maps that can be used to recall stored patterns. Although multiple associative memory models have been proposed, the detailed bifurcation analysis of these models has not been conducted; the analyses conducted on models are often based on empirical approaches. As a result, we focus on the bifurcation analysis of the proposed high dimensional coupled network to propose and verify the reduced model of the Gaussian associative memory model. In the proposed model, successful retrieval of the stored patterns is associated with the existence of stable asymmetric two-periodic points, which are observed in a coupled system of Gaussian maps. In addition, when the Gaussian associative memory recalls the pseudo-patterns, they correspond to high order periodic points and chaotic behavior in the coupled maps.
First, this study discusses the structure of the Gaussian associative memory and further addresses the manner in which its reduced model can be formed. Subsequently, the results of the bifurcation analysis of the reduced model are presented. Finally, we demonstrate the recalling process of the Gaussian associative memory while considering the noisy patterns as the input pattern.
In this section, we introduce the dynamics of the coupled Gaussian maps for an associative memory and its reduced model.
We proposed a Gaussian associative memory model composed of Gaussian maps [
x i ( t + 1 ) = exp ( − α x i ( t ) 2 ) + β + ε p N ∑ j N w i j x j ( t ) , i = 1 , 2 , ⋯ , N , (1)
where α , β , and ε denote parameters; N is the number of Gaussian maps in the network; p is the number of the stored patterns; w i j is the following symmetric auto-associative matrix [
w i j = 1 p ∑ k = 1 p ( 2 ξ i k − 1 ) ( 2 ξ j k − 1 ) , (2)
where ξ i k represents the ith pixel of the kth stored pattern with a discrete value of zero or one. We adopted the stored binary patterns as shown in Figures 1(a)-(d). Figures 1(e)-(g) are the reduced patterns of Figures 1(a)-(d), which would be addressed later. To remember the patterns with 10 × 10 pixels as shown in Figures 1(a)-(d), the Gaussian associative memory is composed of 100 Gaussian maps so that each pixel in the patterns corresponds to each Gaussian map. For the recalling process, when the value of x i is higher (lower) than the threshold, the ith pixel is white (black) in the output pattern. We set the threshold to −0.5 in the experiments conducted.
We considered a reduced model of the Gaussian associative memory to analyze its bifurcation structure. Investigation of the reduced model can easily help in finding the important characteristics of the phenomena observed in the original model. For the following explanation, let us first define the number of each pixel in 10 × 10 and 4 × 4 patterns as shown in
Further, corresponding 100 Gaussian maps in the Gaussian associative memory were classified into 16 groups when we focused on the combination of the pixel values of the stored patterns.
Group | P4 P3 P2 P1 | Pixel No. | hi |
---|---|---|---|
1 | 0 0 0 0 | 5, 6, 15, 61, 84, 85, 93, 94, 95 | 9 |
2 | 0 0 0 1 | 1, 2, 10, 11, 83 | 5 |
3 | 0 0 1 0 | 7, 16, 25, 71 | 4 |
4 | 0 0 1 1 | 24, 33, 55, 56, 57, 67 | 6 |
5 | 0 1 0 0 | 3, 4, 8, 14, 31, 41, 96 | 7 |
6 | 0 1 0 1 | 9, 12, 13, 81, 82, 91, 92 | 7 |
7 | 0 1 1 0 | 17, 21, 32, 48, 58, 97 | 6 |
8 | 0 1 1 1 | 22, 23, 47, 68, 72, 73 | 6 |
9 | 1 0 0 0 | 30, 39, 40, 50, 51, 70, 80, 98 | 8 |
10 | 1 0 0 1 | 20, 29, 89, 90, 99, 100 | 6 |
11 | 1 0 1 0 | 35, 49, 60, 66, 75, 76 | 6 |
12 | 1 0 1 1 | 34, 44, 45, 46, 65, 74 | 6 |
13 | 1 1 0 0 | 42, 52, 69 | 3 |
14 | 1 1 0 1 | 18, 19, 28, 78, 79, 88 | 6 |
15 | 1 1 1 0 | 26, 43, 53, 59, 62, 86, 87 | 7 |
16 | 1 1 1 1 | 27, 36, 37, 38, 54, 63, 64, 77 | 8 |
multiple Gaussian maps in the group could be represented by a representative Gaussian map. As a result, the reduced stored patterns are obtained. Figures 1(e)-(h) show the reduced patterns with 4 × 4 pixels, corresponding to the stored patterns as shown in Figures 1(a)-(d).
The dynamics of the reduced model of the Gaussian associative memory can be described as a difference equation, which can be expressed as follows:
x ( t + 1 ) = f ( x ( t ) ) . (3)
Equivalently, they can be described as an iterated map, which can be expressed as follows:
f : ℝ 16 → ℝ 16 ; x ↦ f ( x ) , (4)
where t denotes discrete time, ℝ represents a set of real numbers, and x and f represent ( x 1 , x 2 , ⋯ , x 16 ) Τ and ( f 1 , f 2 , ⋯ , f 16 ) Τ , respectively. The dynamics of the reduced system of Gaussian associative memory discussed herein are described as
( f 1 ( x ) f 2 ( x ) ⋮ f 16 ( x ) ) = ( exp ( − α x 1 ( t ) 2 ) + β + ε h 1 p 16 ∑ j 16 w 1 j x j ( t ) exp ( − α x 2 ( t ) 2 ) + β + ε h 2 p 16 ∑ j 16 w 2 j x j ( t ) ⋮ exp ( − α x 16 ( t ) 2 ) + β + ε h 16 p 16 ∑ j 16 w 16 j x j ( t ) ) , (5)
where h i is the correction coefficient which is equal to the number of Gaussian maps in each group as shown in the fourth column of
This section presents the method of bifurcation analysis for analyzing the reduced model of the Gaussian associative memory and the necessary conditions and the parameter settings used in the simulation analysis of the Gaussian Associative memory. A notation list for all indices and parameters used in this paper is shown in
In bifurcation analysis, we used a method based on the qualitative bifurcation theory [
x * − f ( x * ) = 0 (6)
becomes a fixed point in Equation (5). The characteristic equation for the fixed point x * is defined as
χ ( x * , m ) = det ( m I − D f ( x * ) ) = 0, (7)
where I is the 16 × 16 identity matrix and D f denotes the derivative of f . We consider x * to be hyperbolic if none of the absolute eigenvalues of D f are at unity. Note that in Equation (6), an m-periodic point can be investigated by replacing f with f m , i.e., the mth iteration of f . In the following discussion, we only consider the properties of a fixed point of f , though a similar argument can be applied to a periodic point of f .
Let us consider the topological classification of a hyperbolic fixed point x * . The topological type of a hyperbolic fixed point is determined by dim E u and det L u , where E u is the intersection of ℝ 16 and the direct sum of the generalized eigenspaces of D f ( x * ) corresponding to eigenvalue m such that | μ i | > 1 , and L u = D f ( x * ) | E u .
When det L u > 0 and det L u < 0 , the hyperbolic fixed point is called D-type and I-type, respectively. Based on this definition, we have 33 topologically different types of hyperbolic fixed points: D k , k = 0 , 1 , ⋯ , 16 and I k , k = 1 , 2 , ⋯ , 16 .
Symbols | Definition |
---|---|
t | discrete time steps |
ℝ | set of real numbers |
x i ( t ) | ith internal state variable at t |
N | number of the Gaussian maps in the coupled network |
w i j | symmetric auto-associative matrix |
p | number of the stored patterns |
α , β , and ε | system parameters of the Gaussian associative memory |
ξ i k | ith pixel of kth input binary pattern with a discrete value of zero or one |
h i | correction coefficient corresponding to ith classified group |
x | ( x 1 , x 2 , ⋯ , x 16 ) Τ |
x * | fixed point of function f |
f | ( f 1 , f 2 , ⋯ , f 16 ) Τ |
D f | Jacobian matrix of function f |
μ | characteristic multipliers or eigenvalues of D f |
I | 16 × 16 Identity matrix |
E u | intersection of ℝ 16 and the direct sum of the generalized eigenspaces of D f ( x * ) |
L u | D f ( x * ) | E u |
0 D | completely stable fixed point |
k D , k > 0 | “directly” unstable fixed point |
k I , k > 0 | “inversely” unstable fixed point |
P1, P2, P3, and P4 | stored patterns with 10 × 10 pixels |
Pr1, Pr2, Pr3, and Pr4 | reduced patterns of stored patterns with 4 × 4 pixels |
G l m | tangent bifurcation of m-periodic point |
I l m | period-doubling bifurcation of m-periodic point |
When we consider the distribution of the characteristic multipliers of Equation (7), D and I correspond to the even and odd numbers, respectively, of the characteristic multipliers on the real axis ( − ∞ , − 1 ) and k represents the number of the characteristic multipliers outside the unit circle on the complex plane. When all characteristic multipliers are in the unit circle, the topological type is 0 D that means completely stable; otherwise, D k , k > 0 and I k , k > 0 represent directly unstable and inversely unstable, respectively.
Bifurcation occurs when the topological type of a fixed point is changed by the varying of a system parameter. In the reduced model, co-dimension-one bifurcations, i.e., tangent and period-doubling bifurcations, are observed when hyperbolicity of the system is destroyed. This corresponds to the critical distribution of the characteristic multiplier μ such that μ = + 1 for tangent bifurcation and μ = − 1 for period-doubling bifurcation.
The bifurcation sets of a fixed point were computed by solving the simultaneous Equations (6) and (7). For numerical determination [
With respect to the Gaussian associative memory defined by Equations (1) and (2), we evaluated the recalling ability of the Gaussian associated memory for the 10 × 10 stored patterns as shown in
We investigated the relationship between the number of the inverted bits included in the noisy pattern and recalling probability in the simulation conducted. With each noisy pattern containing 0 to 100 inverted bits, the Gaussian associative memory model was run for 100 iterations. When the output pattern set at t = 100 was exactly same to the stored pattern, we added one to a variable S. After executing for 1000 times executions, the average value of the recalling probability was calculated as S/1000.
In
We investigated the bifurcation sets of the reduced network of Gaussian maps for associative memory on the ( β , ε ) -plane using bifurcation analysis as explained in Section 3.1. In the bifurcation diagrams, we use the following symbols:
G l m tangent bifurcation of the m-periodic point.
I l m period-doubling bifurcation of the m-periodic point.
where l distinguishes the same types of the bifurcation sets of the m-periodic points.
Figures 5-7 represent the bifurcation structures of the fixed points on the ( β , ε ) -plane, which were observed in the reduced model. In
In
of the asymmetric two-periodic points can be separated by setting a threshold so that the Gaussian associative memory model can generate output patterns identical to the stored patterns. In the shaded regions in the bifurcation diagrams shown in
The existence of the stable asymmetric two-periodic points is the key for
making the Gaussian associative model to work effectively. At the same time, the coupled system also has the stable symmetric and asymmetric fixed points. Nevertheless, the stable symmetric fixed point is not appropriate for the associative memory because the output values of each Gaussian map cannot be separated into binary values. In contrast, the asymmetric fixed point seems to be appropriate for the associative memory. However, as shown in
To investigate the generation mechanism of the asymmetric two-periodic points, we focused on the results obtained when the pattern shown in
We demonstrate the retrieval process of the Gaussian associative memory by using the Gaussian associative memory for 10 × 10 stored patterns as shown in
adding the function for avoiding chaotic attractors and high order periodic points to memory.
A novel associative memory model based on the Gaussian coupled maps was proposed. We explored the characteristics of the Gaussian associative memory by investigating its behavior as the number of coupled maps is reduced. When 10 × 10 pixel patterns were stored into the associative memory, each map corresponding to the input pixel was classified into 16 groups based on their synchronization, therefore, we can consider the model as a 16-coupled Gaussian map. The Gaussian associate memory can be simplified by reducing the number of coupled maps which makes it possible to investigate its bifurcation structure by analyzing the reduced model. Based on the results of the analysis, we found the parameter region where the stable asymmetric two-periodic points occurred. The stable asymmetric two-periodic points were appropriate in generating the output patterns corresponding to the binary stored patterns because the trajectories can effectively be separated by setting a threshold. In addition, because the symmetric fixed point did not appear in the parameter region, it is preferable for recalling the stored patterns. We demonstrated the retrieval process of the proposed model by conducting a simulation which showed that the Gaussian associative memory could retrieve stored patterns from the noisy pattern including 30% different pixels.
Hence, it can be concluded that the Gaussian associative memory can efficiently recall stored patterns. However, pseudo-patterns were generated with input patterns having high noise rates. The trajectories associated with those pseudo-patterns were high order periodic points or chaotic behavior. Our future work would focus on investigating methods for preventing the generation of those pseudo-patterns, and to how to enlarge the capacity of the memory. For the former task, the method for avoiding chaos or high order periodic points must work well in preventing the generation of pseudo-patterns too. For the latter, the basin of the attractors and more precise bifurcation analysis would be investigated.
The authors declare no conflicts of interest regarding the publication of this paper.
Kobayashi, M. and Yoshinaga, T. (2019) Bifurcation Analysis of Reduced Network Model of Coupled Gaussian Maps for Associative Memory. International Journal of Modern Nonlinear Theory and Application, 8, 1-16. https://doi.org/10.4236/ijmnta.2019.81001