Mixed Materials
Quantitative Attractor Analysis of High-Capacity Kernel Hopfield Networks
| Title: | Quantitative Attractor Analysis of High-Capacity Kernel Hopfield Networks |
|---|---|
| Additional Titles: | Kernel-based learning methods such as Kernel Logistic Regression (KLR) can substantially increase the storage capacity of Hopfield networks, but the principles governing their performance and stability remain largely uncharacterized. This paper presents a comprehensive quantitative analysis of the attractor landscape in KLR-trained networks to establish a solid foundation for their design and application. Through extensive, statistically validated simulations, we address critical questions of generality, scalability, and robustness. Our comparative analysis shows that KLR and Kernel Ridge Regression (KRR) exhibit similarly high storage capacities and clean attractor landscapes under typical operating conditions, suggesting that this behavior is a general property of kernel regression methods, although KRR is computationally much faster. We identify a non-trivial, scale-dependent law for the kernel width $γ$, demonstrating that optimal capacity requires $γ$ to be scaled such that $γN$ increases with network size $N$. This finding implies that larger networks require more localized kernels, in which each pattern's influence is more spatially confined, to mitigate inter-pattern interference. Under this optimized scaling, we provide clear evidence that storage capacity scales linearly with network size~($P \propto N$). Furthermore, our sensitivity analysis shows that performance is remarkably robust with respect to the choice of the regularization parameter $λ$. Collectively, these findings provide a concise set of empirical principles for designing high-capacity and robust associative memories and clarify the mechanisms that enable kernel methods to overcome the classical limitations of Hopfield-type models. |
| Authors: | Tamamori, Akira |
| Publisher Information: | 2025-05-02 2026-01-09 |
| Document Type: | Mixed materials |
| Abstract: | 16 pages, 7 figures; submitted to NOLTA, IEICE |
| Index Terms: | Machine Learning, Neural and Evolutionary Computing, text |
| URL: | |
| Availability: | Open access content. Open access content |
| Other Numbers: | COO oai:arXiv.org:2505.01218 1577677130 |
| Contributing Source: | CORNELL UNIV From OAIster®, provided by the OCLC Cooperative. |
| Accession Number: | edsoai.on1577677130 |
| Database: | OAIster |
| Description not available. |