1. Introduction

Previous studies have proposed multiple-associative memory models based on dynamical systems [1] [2] [3] [4] . J. J. Hopfield proposed a neural network model which is popularly referred to as an associative memory [5] . The Hopfield model functions as an associative memory that mimics the human brain. Hebbian learning can be used to calculate the weight of connections between neurons using stored patterns. In the recalling process of the Hopfield model, the system needs to be iteratively updated until its energy function, which is guaranteed to decrease as updating continues, converges to a local minimum. Because the local minimum corresponds to a stored pattern, the Hopfield associative memory can recall one of the stored patterns depending on the initial values of the network. This is called auto-association.

Hopfield model has been extended to develop bidirectional and multidirectional associative memories [6] [7] [8] [9] . The bidirectional associative memory comprises a two-layer network and the model can recall another pattern by using input pattern. This is referred to as hetero-association. The size of the recalling pattern can be different from that of the stored pattern. On the other hand, the multidirectional associative memory has an extended structure of the bidirectional model with three or more layers. The associative structure of multidirectional model is more complicated than the bidirectional model, and the model can successively recall another pattern as it is being iteratively updated. In contrast to the conventional associative models, a chaotic associative memory model, which is made up of chaotic neurons, has been proposed as a non-periodic associative memory [10] . While the conventional models adopted the Hebbian learning in calculating the weight between neurons, it incorporates an auto-association matrix into the chaotic neuron associative memory model. One of the significant characteristics of chaotic associative memory is that it can successively recall stored patterns with transforming the output patterns depending on the system parameters and the chaotic neuron dynamics. Another characteristic of the chaos associative memory is that it can recall not only stored patterns but also reverse patterns of the stored patterns.

The retrieval ability and the storage capacity are important points to be considered when evaluating associative memory models and have been the subject of many studies. One major problem associated with the associative memory model is the generation of the pseudo-patterns. How to avoid the generation of the pseudo-patterns has received remarkable attention in associative memory research community.

However, the mechanisms that are used to effectively recall the stored patterns and to generate pseudo-patterns have not been well elucidated. Herein, we propose a novel associative memory model comprised of Gaussian map to investigate the mechanism of the associative memory based on the qualitative bifurcation theory.

The Gaussian map is a one-dimensional dynamical system which generates various phenomena including periodic points and chaos [11] [12] . Asymmetric fixed and periodic points are observed under the particular parameter settings when investigating the dynamics of the coupled Gaussian maps. By associating the asymmetric values of the coupled Gaussian maps with the binary values of the recalling patterns, the Gaussian associative memory can successfully be applied in the recalling process as well as the other conventional methods. The Gaussian associative memory has a simple structure as the Gaussian map is a one-dimensional dynamical system and shows very complicated phenomena such as chaotic behavior and quasi-periodic oscillations. Furthermore, the bifurcation structure of the associative memory can be qualitatively analyzed by considering the reduced model of the Gaussian associative memory.

This study investigates the bifurcation structure of periodic points observed in the high dimensional coupled network of the Gaussian maps that can be used to recall stored patterns. Although multiple associative memory models have been proposed, the detailed bifurcation analysis of these models has not been conducted; the analyses conducted on models are often based on empirical approaches. As a result, we focus on the bifurcation analysis of the proposed high dimensional coupled network to propose and verify the reduced model of the Gaussian associative memory model. In the proposed model, successful retrieval of the stored patterns is associated with the existence of stable asymmetric two-periodic points, which are observed in a coupled system of Gaussian maps. In addition, when the Gaussian associative memory recalls the pseudo-patterns, they correspond to high order periodic points and chaotic behavior in the coupled maps.

First, this study discusses the structure of the Gaussian associative memory and further addresses the manner in which its reduced model can be formed. Subsequently, the results of the bifurcation analysis of the reduced model are presented. Finally, we demonstrate the recalling process of the Gaussian associative memory while considering the noisy patterns as the input pattern.

2. Model Description

In this section, we introduce the dynamics of the coupled Gaussian maps for an associative memory and its reduced model.

2.1. Coupled Gaussian Maps for Associative Memory

We proposed a Gaussian associative memory model composed of Gaussian maps [13] . The dynamics of ith Gaussian map in the Gaussian associative memory can be described as follows:

$x_i\left(t+1\right)=\exp \left(-\alpha \mathrm{<mi>\; </mi>}{x}_i{\left(t\right)}^2\right)+\beta +\epsilon \frac{p}{N}{\displaystyle \underset{j}{\overset{N}{\sum }}}{w}_{ij}{x}_j\left(t\right),\mathrm{<mi>\; </mi>}i=1,2,\cdots ,N,$ (1)

where $\alpha$ , $\beta$ , and $\epsilon$ denote parameters; N is the number of Gaussian maps in the network; p is the number of the stored patterns; $w_{ij}$ is the following symmetric auto-associative matrix [10] :

$w_{ij}=\frac{1}{p}{\displaystyle \underset{k=1}{\overset{p}{\sum }}}\left(2{\xi }_i^k-1\right)\left(2{\xi }_j^k-1\right),$ (2)

where ${\xi }_i^k$ represents the ith pixel of the kth stored pattern with a discrete value of zero or one. We adopted the stored binary patterns as shown in Figures 1(a)-(d). Figures 1(e)-(g) are the reduced patterns of Figures 1(a)-(d), which would be addressed later. To remember the patterns with 10 × 10 pixels as shown in Figures 1(a)-(d), the Gaussian associative memory is composed of 100 Gaussian maps so that each pixel in the patterns corresponds to each Gaussian map. For the recalling process, when the value of $x_i$ is higher (lower) than the threshold, the ith pixel is white (black) in the output pattern. We set the threshold to −0.5 in the experiments conducted.

2.2. Reduced Model of Gaussian Associative Memory

We considered a reduced model of the Gaussian associative memory to analyze its bifurcation structure. Investigation of the reduced model can easily help in finding the important characteristics of the phenomena observed in the original model. For the following explanation, let us first define the number of each pixel in 10 × 10 and 4 × 4 patterns as shown in Figure 2. The number of the pixel increases from the top-left to the bottom-right corners.

Further, corresponding 100 Gaussian maps in the Gaussian associative memory were classified into 16 groups when we focused on the combination of the pixel values of the stored patterns. Table 1 shows the classification of each Gaussian map in 10 × 10 Gaussian associative memory. The first and the second columns represent the number of the group and the combinations of the pixel values of each stored pattern, respectively. When one of the stored patterns is used as the initial pattern on the recalling process, $x_i\left(0\right)$ in the same group is identical. In this situation, $w_{ij}$ becomes the same in the group. Figure 3 shows the classification of 10 × 10 pixels which correspond to the Gaussian maps. The Gaussian maps in the same group oscillate in-phase or converge to the same fixed point. When the Gaussian maps in the same group were synchronized,

multiple Gaussian maps in the group could be represented by a representative Gaussian map. As a result, the reduced stored patterns are obtained. Figures 1(e)-(h) show the reduced patterns with 4 × 4 pixels, corresponding to the stored patterns as shown in Figures 1(a)-(d).

The dynamics of the reduced model of the Gaussian associative memory can be described as a difference equation, which can be expressed as follows:

$x\left(t+1\right)=f\left(x\left(t\right)\right)\mathrm{.}$ (3)

Equivalently, they can be described as an iterated map, which can be expressed as follows:

$f\mathrm{<mo>\; :\; </mo>}{ℝ}^{16}\to {ℝ}^{16}\mathrm{<mo>\; ;\; </mo>}x\mapsto f\left(x\right)\mathrm{,}$ (4)

where t denotes discrete time, $ℝ$ represents a set of real numbers, and $x$ and $f$ represent ${\left(x_1\mathrm{,}{x}_2\mathrm{,}\cdots \mathrm{,}{x}_{16}\right)}^{{\rm T}}$ and ${\left(f_1\mathrm{,}{f}_2\mathrm{,}\cdots \mathrm{,}{f}_{16}\right)}^{{\rm T}}$ , respectively. The dynamics of the reduced system of Gaussian associative memory discussed herein are described as

$\left(\begin{array}{c}{f}_1\left(x\right)\\ f_2\left(x\right)\\ ⋮\\ f_{16}\left(x\right)\end{array}\right)=\left(\begin{array}{c}\exp \left(-\alpha x_1{\left(t\right)}^2\right)+\beta +\epsilon h_1\frac{p}{16}{\displaystyle \underset{j}{\overset{16}{\sum }}}{w}_{1j}{x}_j\left(t\right)\\ \exp \left(-\alpha x_2{\left(t\right)}^2\right)+\beta +\epsilon h_2\frac{p}{16}{\displaystyle \underset{j}{\overset{16}{\sum }}}{w}_{2j}{x}_j\left(t\right)\\ ⋮\\ \exp \left(-\alpha x_{16}{\left(t\right)}^2\right)+\beta +\epsilon h_{16}\frac{p}{16}{\displaystyle \underset{j}{\overset{16}{\sum }}}{w}_{16j}{x}_j\left(t\right)\end{array}\right)\mathrm{,}$ (5)

where $h_i$ is the correction coefficient which is equal to the number of Gaussian maps in each group as shown in the fourth column of Table 1.

3. Method

This section presents the method of bifurcation analysis for analyzing the reduced model of the Gaussian associative memory and the necessary conditions and the parameter settings used in the simulation analysis of the Gaussian Associative memory. A notation list for all indices and parameters used in this paper is shown in Table 2.

3.1. Bifurcation Analysis

In bifurcation analysis, we used a method based on the qualitative bifurcation theory [14] . The point $x^{\mathrm{<mo>\; *\; </mo>}}$ satisfying

$x^{\mathrm{<mo>\; *\; </mo>}}-f\left(x^{\mathrm{<mo>\; *\; </mo>}}\right)=0$ (6)

becomes a fixed point in Equation (5). The characteristic equation for the fixed point $x^{\mathrm{<mo>\; *\; </mo>}}$ is defined as

$\chi \left(x^{\mathrm{<mo>\; *\; </mo>}}\mathrm{,}m\right)=\det \left(mI-Df\left(x^{\mathrm{<mo>\; *\; </mo>}}\right)\right)=\mathrm{0,}$ (7)

where $I$ is the 16 × 16 identity matrix and $Df$ denotes the derivative of $f$ . We consider $x^{\mathrm{<mo>\; *\; </mo>}}$ to be hyperbolic if none of the absolute eigenvalues of $Df$ are at unity. Note that in Equation (6), an m-periodic point can be investigated by replacing $f$ with $f^m$ , i.e., the mth iteration of $f$ . In the following discussion, we only consider the properties of a fixed point of $f$ , though a similar argument can be applied to a periodic point of $f$ .

Let us consider the topological classification of a hyperbolic fixed point $x^{\mathrm{<mo>\; *\; </mo>}}$ . The topological type of a hyperbolic fixed point is determined by $\dim E^u$ and $\det L^u$ , where $E^u$ is the intersection of ${ℝ}^{16}$ and the direct sum of the generalized eigenspaces of $Df\left(x^{\mathrm{<mo>\; *\; </mo>}}\right)$ corresponding to eigenvalue $m$ such that $\left|{\mu }_i\right|>1$ , and $L^u={Df\left(x^{\mathrm{<mo>\; *\; </mo>}}\right)|}_{E^u}$ .

When $\det L^u>0$ and $\det L^u<0$ , the hyperbolic fixed point is called D-type and I-type, respectively. Based on this definition, we have 33 topologically different types of hyperbolic fixed points: ${}_{k}D,\mathrm{<mi>\; </mi>}k=0,1,\cdots ,16$ and ${}_{k}I,\mathrm{<mi>\; </mi>}k=1,2,\cdots ,16$ .

When we consider the distribution of the characteristic multipliers of Equation (7), D and I correspond to the even and odd numbers, respectively, of the characteristic multipliers on the real axis $\left(-\infty \mathrm{,}-1\right)$ and k represents the number of the characteristic multipliers outside the unit circle on the complex plane. When all characteristic multipliers are in the unit circle, the topological type is ${}_{0}D$ that means completely stable; otherwise, ${}_{k}D,\mathrm{<mi>\; </mi>}k>0$ and ${}_{k}I,\mathrm{<mi>\; </mi>}k>0$ represent directly unstable and inversely unstable, respectively.

Bifurcation occurs when the topological type of a fixed point is changed by the varying of a system parameter. In the reduced model, co-dimension-one bifurcations, i.e., tangent and period-doubling bifurcations, are observed when hyperbolicity of the system is destroyed. This corresponds to the critical distribution of the characteristic multiplier $\mu$ such that $\mu =+1$ for tangent bifurcation and $\mu =-1$ for period-doubling bifurcation.

The bifurcation sets of a fixed point were computed by solving the simultaneous Equations (6) and (7). For numerical determination [14] , we used Newton-Raphson method. The Jacobian matrix of the set of equations was derived from the first and second derivatives of map f.

3.2. Simulation Settings

With respect to the Gaussian associative memory defined by Equations (1) and (2), we evaluated the recalling ability of the Gaussian associated memory for the 10 × 10 stored patterns as shown in Figure 1. The parameters were set as follows: $\alpha =16$ , $\beta =-0.5$ and $\epsilon =0.7$ in Equation (1) at $p=4$ and $N=100$ . The white and black pixels of an initial input pattern were converted to $x_i\left(0\right)=0$ and −1, respectively. As an initial input pattern, we used the noisy pattern which includes inverted bits compared with the original stored pattern as shown in Figures 1(a)-(d). When the noisy pattern includes no inverted bits, the pattern is completely the same as the stored pattern. When we increase the ratio of the noise added in the initial pattern, the noisy pattern finally becomes the completely inverted pattern of the original stored pattern.

We investigated the relationship between the number of the inverted bits included in the noisy pattern and recalling probability in the simulation conducted. With each noisy pattern containing 0 to 100 inverted bits, the Gaussian associative memory model was run for 100 iterations. When the output pattern set at $t=100$ was exactly same to the stored pattern, we added one to a variable S. After executing for 1000 times executions, the average value of the recalling probability was calculated as S/1000.

4. Results of Bifurcation Analysis

Figure 4 shows typical fixed points observed in the reduced model. The fixed points can be obtained by setting initial values arbitrarily to $x_i\left(0\right)$ and iteratively updating the values of $x_i\left(t\right)$ according to Equation (5).

In Figure 4(a), each value of $x_i$ is slightly different from one other. That is because the coefficients represented by $w_{ij}$ and $h_i$ represent the pixel values of the stored patterns while the number of Gaussian maps is represented by one reduced map, respectively so that the reduced model does not have complete symmetricity. Therefore, $x_i$ described in Equation (5) becomes different from each other. However, to distinguish the fixed points shown in Figure 4(a) and Figure 4(b), let us call them symmetric and asymmetric fixed points, respectively.

We investigated the bifurcation sets of the reduced network of Gaussian maps for associative memory on the $\left(\beta \mathrm{,}\epsilon \right)$ -plane using bifurcation analysis as explained in Section 3.1. In the bifurcation diagrams, we use the following symbols:

$G_l^m$ tangent bifurcation of the m-periodic point.

$I_l^m$ period-doubling bifurcation of the m-periodic point.

where l distinguishes the same types of the bifurcation sets of the m-periodic points.

4.1. Bifurcation Structure of Fixed Points

Figures 5-7 represent the bifurcation structures of the fixed points on the $\left(\beta \mathrm{,}\epsilon \right)$ -plane, which were observed in the reduced model. In Figure 5, the stable symmetric fixed point was observed in the region represented by . In contrast, the asymmetric fixed points were observed in the region between $I_1^1$ and $G_2^1$ represented by as shown in Figure 5, and the shaded regions in Figure 6 and Figure 7. Figures 8(a)-(c) represent the output patterns of 4 × 4 pixels obtained using the asymmetric fixed points shown in Figures 5-7, respectively. Figures 8(d)-(f) correspond to the 10 × 10 pixel patterns of Figures 8(a)-(c). The asymmetric fixed point that appears in Figure 6 is the only the fixed point that represents exactly the same pattern represented by one of the stored patterns. The rest are the pseudo-patterns. Note that the asymmetric fixed points corresponding to other stored patterns also exist in approximately the same parameter regions, however, they were omitted here.

In Figure 6, in the right-hand side of the parameter region where the stable asymmetric fixed point exits, multiple period doubling bifurcations appeared, which were represented by $I_1^1$ to $I_{16}^1$ in a clockwise direction. By going through the period doubling bifurcations with a changing parameter, the asymmetric fixed point became unstable. After passing through eight period doubling bifurcations from $I_1^1$ to $I_8^1$ , the asymmetric fixed point lost its stability and became ${}_{8}D$ . The ends of the 16 curves of period doubling bifurcations converged at $\left(\beta \mathrm{,}\epsilon \right)\simeq \left(-\mathrm{0.95223,0.0}\right)$ . The reduced model loses its coupled structure when $\epsilon =0$ .

4.2. Bifurcation Structure of Two-Periodic Points

Figure 9 shows the different types of waveforms of stable asymmetric two-periodic points when the pattern of Figure 1(e) was used as an initial pattern at $\beta =-0.4$ and $\alpha =16$ . The waveforms of Figures 9(a)-(c) were observed at $\epsilon =0.6$ , 0.75 and 0.8. At $\epsilon =0.6$ , the two-periodic points were in-phase while the phases were shifted at $\epsilon =0.75$ and $\epsilon =0.8$ .

Figure 10 shows the bifurcation diagrams of two-periodic points observed in the reduced model as each stored pattern in Figures 1(e)-(h) was used as an initial pattern. In the Gaussian associative memory model, the oscillatory responses

of the asymmetric two-periodic points can be separated by setting a threshold so that the Gaussian associative memory model can generate output patterns identical to the stored patterns. In the shaded regions in the bifurcation diagrams shown in Figure 10, the regions where stable asymmetric two-periodic points exist can be seen as partly over-wrapped. The shapes of the regions where stable asymmetric two-periodic points appear are similar to the downward triangles, and the regions are surrounded by period doubling and tangent bifurcations at the left and right-hand sides, respectively.

The existence of the stable asymmetric two-periodic points is the key for

making the Gaussian associative model to work effectively. At the same time, the coupled system also has the stable symmetric and asymmetric fixed points. Nevertheless, the stable symmetric fixed point is not appropriate for the associative memory because the output values of each Gaussian map cannot be separated into binary values. In contrast, the asymmetric fixed point seems to be appropriate for the associative memory. However, as shown in Figure 6 and Figure 7, the parameter region where the stable asymmetric fixed point appears over-wraps the region where the stable symmetric fixed point exists. We cannot choose the parameter values in such parameter regions because the trajectories might be attracted to the symmetric fixed points. Hence, we decided to set the parameter values to $\alpha =16$ , $\beta =-0.5$ , and $\epsilon =0.7$ for simulating the recalling process of the Gaussian associative memory.

To investigate the generation mechanism of the asymmetric two-periodic points, we focused on the results obtained when the pattern shown in Figure 1(f) was used as an initial pattern. In Figure 10(b), the stable asymmetric two-periodic point that exists in the region surrounded by $I_1^2$ and $G_1^2$ was derived from the period doubling bifurcation of the stable asymmetric fixed point, represented by $I_{16}^1$ as shown in Figure 6. On the other hand, the stable asymmetric two-periodic points between $I_4^2$ and $G_4^2$ in Figure 10(b) was generated from $I_1^1$ as shown in Figure 6. Based on the results obtained, we found that the other stable asymmetric two-periodic points can also be caused by a stable asymmetric fixed point.

5. Simulation

We demonstrate the retrieval process of the Gaussian associative memory by using the Gaussian associative memory for 10 × 10 stored patterns as shown in Figure 1. In Figures 11(a)-(d), the initial patterns at $t=0$ include 30 pixels of the inverted bits compared with the stored patterns. In each case, the Gaussian associative memory could recall the corresponding stored pattern at $t=2$ . In contrast, when the initial pattern includes higher rate inverted bits, they converged to the reverse pattern of the corresponding stored pattern. When the rate of the noise in the initial pattern is around 50%, pseudo-patterns rarely appeared. Figure 11(e) shows the output sequence of a pseudo pattern recalled from the initial pattern shown at $t=0$ . Based on the result of the simulation experiments conducted, we found that the pseudo-patterns generation relates to the existence of high order periodic points or chaotic attractors. The waveform shown in Figure 12 corresponds to the output sequence of Figure 11(e). Generation of pseudo-patterns in a Gaussian associative memory can be prevented by

adding the function for avoiding chaotic attractors and high order periodic points to memory.

Figure 13(a) and Figure 13(b) represent the recalling probability of noisy patterns made from a stored pattern P₁ as shown in Figure 1(a) on the Gaussian and Hopfield associative memory models, respectively. In Figure 13(a) and Figure 13(b), the retrieval ability was 100% up to 20 inverted bits, however, the retrieval ability gradually declined as the number of inverted bits increased. The curve with the title “Others” in the diagram represents the recalling probability of the pseudo-patterns. The curves for the recalling probability of pseudo-patterns have two peaks at around 40 and 60 inverted bits. Comparing the Gaussian and Hopfield associative memory, the recalling probability of pseudo-patterns in the Gaussian associative memory is lower than that of Hopfield associative memory. In contrast, the recalling probability of other stored patterns of the Gaussian associative memory is higher than that of Hopfield associative memory at around 50 inverted bits. In terms of pseudo-pattern generation, the Gaussian associative memory outperformed Hopfield associative memory.

6. Conclusions

A novel associative memory model based on the Gaussian coupled maps was proposed. We explored the characteristics of the Gaussian associative memory by investigating its behavior as the number of coupled maps is reduced. When 10 × 10 pixel patterns were stored into the associative memory, each map corresponding to the input pixel was classified into 16 groups based on their synchronization, therefore, we can consider the model as a 16-coupled Gaussian map. The Gaussian associate memory can be simplified by reducing the number of coupled maps which makes it possible to investigate its bifurcation structure by analyzing the reduced model. Based on the results of the analysis, we found the parameter region where the stable asymmetric two-periodic points occurred. The stable asymmetric two-periodic points were appropriate in generating the output patterns corresponding to the binary stored patterns because the trajectories can effectively be separated by setting a threshold. In addition, because the symmetric fixed point did not appear in the parameter region, it is preferable for recalling the stored patterns. We demonstrated the retrieval process of the proposed model by conducting a simulation which showed that the Gaussian associative memory could retrieve stored patterns from the noisy pattern including 30% different pixels.

Hence, it can be concluded that the Gaussian associative memory can efficiently recall stored patterns. However, pseudo-patterns were generated with input patterns having high noise rates. The trajectories associated with those pseudo-patterns were high order periodic points or chaotic behavior. Our future work would focus on investigating methods for preventing the generation of those pseudo-patterns, and to how to enlarge the capacity of the memory. For the former task, the method for avoiding chaos or high order periodic points must work well in preventing the generation of pseudo-patterns too. For the latter, the basin of the attractors and more precise bifurcation analysis would be investigated.