Dissertation/ Thesis
Optimal Learning Rates for Neural Networks
| العنوان: | Optimal Learning Rates for Neural Networks |
|---|---|
| المؤلفون: | Moncur, Tyler |
| المصدر: | Theses and Dissertations. |
| بيانات النشر: | BYU ScholarsArchive. |
| المجموعة: | Brigham Young University |
| مصطلحات موضوعية: | neural network, learning rate, crelu, Physical Sciences and Mathematics |
| الوصف: | Neural networks have long been known as universal function approximators and have more recently been shown to be powerful and versatile in practice. But it can be extremely challenging to find the right set of parameters and hyperparameters. Model training is both expensive and difficult due to the large number of parameters and sensitivity to hyperparameters such as learning rate and architecture. Hyperparameter searches are notorious for requiring tremendous amounts of processing power and human resources. This thesis provides an analytic approach to estimating the optimal value of one of the key hyperparameters in neural networks, the learning rate. Where possible, the analysis is computed exactly, and where necessary, approximations and assumptions are used and justified. The result is a method that estimates the optimal learning rate for a certain type of network, a fully connected CReLU network. |
| Original Identifier: | oai:scholarsarchive.byu.edu:etd-9662 |
| نوع الوثيقة: | Text |
| وصف الملف: | application/pdf |
| الاتاحة: | https://scholarsarchive.byu.edu/etd/8662 https://scholarsarchive.byu.edu/cgi/viewcontent.cgi?article=9662&context=etd |
| Rights: | URL: https://lib.byu.edu/about/copyright/ |
| رقم الانضمام: | edsndl.BGMYU2.oai.scholarsarchive.byu.edu.etd.9662 |
| قاعدة البيانات: | Networked Digital Library of Theses & Dissertations |
| الوصف غير متاح. |