Academic Journal
Revisiting deep information propagation: Fractal frontier and finite-size effects.
| Τίτλος: | Revisiting deep information propagation: Fractal frontier and finite-size effects. |
|---|---|
| Συγγραφείς: | D'Inverno GA; MathLab, International School for Advanced Studies (SISSA), Via Bonomea 265, Trieste, 34136, Italy. Electronic address: gdinvern@sissa.it., Hu Z; Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, Station 17, Lausanne, 1015, Switzerland. Electronic address: zhiyuan.hu@epfl.ch., Davy L; Laboratoire de Physique de l'ENS Lyon, CNRS UMR 5672, 46 All. d'Italie, Lyon, 69007, France. Electronic address: leo.davy@ens-lyon.fr., Unser M; Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, Station 17, Lausanne, 1015, Switzerland. Electronic address: michael.unser@epfl.ch., Rozza G; MathLab, International School for Advanced Studies (SISSA), Via Bonomea 265, Trieste, 34136, Italy. Electronic address: grozza@sissa.it., Dong J; Biomedical Imaging Group, École Polytechnique Fédérale de Lausanne, Station 17, Lausanne, 1015, Switzerland. Electronic address: jonathan.dong@epfl.ch. |
| Πηγή: | Neural networks : the official journal of the International Neural Network Society [Neural Netw] 2026 Oct; Vol. 202, pp. 109013. Date of Electronic Publication: 2026 Apr 16. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: Pergamon Press Country of Publication: United States NLM ID: 8805018 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1879-2782 (Electronic) Linking ISSN: 08936080 NLM ISO Abbreviation: Neural Netw Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: New York : Pergamon Press, [c1988- |
| Ιατρικοί όροι (MeSH): | Fractals* , Neural Networks, Computer*, Convolutional Neural Networks ; Nonlinear Dynamics ; Fourier Analysis ; Multilayer Perceptrons ; Algorithms |
| Περίληψη: | Information propagation characterizes how input correlations evolve across layers in deep neural networks. This framework has been well studied using mean-field theory, which assumes infinitely wide networks. However, these assumptions break down for practical, finite-size networks. In this work, we study information propagation in randomly initialized neural networks with finite width and reveal that the boundary between ordered and chaotic regimes exhibits a fractal structure. This shows the fundamental complexity of neural network dynamics, in a setting that is independent of input data and optimization. To extend this analysis beyond multilayer perceptrons, we leverage recently introduced Fourier-based structured transforms, and show that information propagation in convolutional neural networks also follow the same behavior. In practice, our investigation highlights the importance of finite network depth with respect to the tradeoff between separation and robustness. We also show that fractal patterns are observed for information propagation in the backward pass, i.e., backpropagation from the last to the first layer of finite-size networks. (Copyright © 2026 Elsevier Ltd. All rights reserved.) |
| Competing Interests: | Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. |
| Contributed Indexing: | Keywords: Finite-size effects; Fractal dimension; Information propagation |
| Entry Date(s): | Date Created: 20260422 Date Completed: 20260718 Latest Revision: 20260718 |
| Update Code: | 20260718 |
| DOI: | 10.1016/j.neunet.2026.109013 |
| PMID: | 42019214 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 1879-2782 |
|---|---|
| DOI: | 10.1016/j.neunet.2026.109013 |