Report
Convergence and scaling of Boolean-weight optimization for hardware reservoirs
| العنوان: | Convergence and scaling of Boolean-weight optimization for hardware reservoirs |
|---|---|
| المؤلفون: | Andreoli, Louis, Chrétien, Stéphane, Porte, Xavier, Brunner, Daniel |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Statistics - Machine Learning, Computer Science - Machine Learning |
| الوصف: | Hardware implementation of neural network are an essential step to implement next generation efficient and powerful artificial intelligence solutions. Besides the realization of a parallel, efficient and scalable hardware architecture, the optimization of the system's extremely large parameter space with sampling-efficient approaches is essential. Here, we analytically derive the scaling laws for highly efficient Coordinate Descent applied to optimizing the readout layer of a random recurrently connection neural network, a reservoir. We demonstrate that the convergence is exponential and scales linear with the network's number of neurons. Our results perfectly reproduce the convergence and scaling of a large-scale photonic reservoir implemented in a proof-of-concept experiment. Our work therefore provides a solid foundation for such optimization in hardware networks, and identifies future directions that are promising for optimizing convergence speed during learning leveraging measures of a neural network's amplitude statistics and the weight update rule. Comment: Submitted to ECML-PKDD workshop Deep Learning meets Neuromorphic Hardware |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2305.07908 |
| رقم الانضمام: | edsarx.2305.07908 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |