Academic Journal
In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
| Title: | In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting |
|---|---|
| Authors: | Philippenko, Constantin, Scaman, Kevin, Massoulié, Laurent |
| Contributors: | Philippenko, Constantin |
| Source: | Proceedings of the AAAI Conference on Artificial Intelligence. 39:19904-19912 |
| Publication Status: | Preprint |
| Publisher Information: | Association for the Advancement of Artificial Intelligence (AAAI), 2025. |
| Publication Year: | 2025 |
| Subject Terms: | Optimization, Matrix factorisation, Machine Learning, FOS: Computer and information sciences, Optimization and Control (math.OC), Optimization and Control, Federated learning, FOS: Mathematics, [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG], [STAT.ML] Statistics [stat]/Machine Learning [stat.ML], Machine Learning (cs.LG) |
| Description: | This work presents a novel approach to low-rank matrix factorization in a federated learning context, where multiple clients collaboratively solve a matrix decomposition problem without sharing their local data. The algorithm introduces a power initialization technique for the global factorization matrix and combines it with local gradient descent updates to achieve strong theoretical and practical guarantees. Considering this power initialization, we rewrite the previous smooth non-convex problem into a smooth strongly-convex problem that we solve using a parallel Nesterov gradient descent potentially requiring a single step of communication at the initialization step. We provide a linear rate of convergence of the excess loss, our results improve the rates of convergence given in the literature. We provide an upper bound on the Frobenius-norm error of reconstruction under the power initialization strategy. We complete our analysis with experiments on both synthetic and real data. |
| Document Type: | Article Conference object |
| File Description: | application/pdf |
| ISSN: | 2374-3468 2159-5399 |
| DOI: | 10.1609/aaai.v39i19.34192 |
| DOI: | 10.48550/arxiv.2409.08771 |
| Access URL: | http://arxiv.org/abs/2409.08771 https://hal.science/hal-04893622v2 |
| Rights: | CC BY |
| Accession Number: | edsair.doi.dedup.....43f1eae6b3e8dfb7e16e7bd3ff8606d2 |
| Database: | OpenAIRE |
| ISSN: | 23743468 21595399 |
|---|---|
| DOI: | 10.1609/aaai.v39i19.34192 |