Academic Journal
Deep Model Compression: Distilling Knowledge from Noisy Teachers
| العنوان: | Deep Model Compression: Distilling Knowledge from Noisy Teachers |
|---|---|
| المؤلفون: | Sau, B B, Balasubramanian, Vineeth N |
| بيانات النشر: | Cornell University Library |
| سنة النشر: | 2016 |
| المجموعة: | Research Archive of Indian Institute of Technology, Hyderabad (RAIITH) |
| مصطلحات موضوعية: | Big Data Analytics |
| الوصف: | The remarkable successes of deep learning models across various applications have resulted in the design of deeper networks that can solve complex problems. How- ever, the increasing depth of such models also results in a higher storage and runtime complexity, which restricts the deployability of such very deep models on mobile and portable devices, which have limited storage and battery capacity. While many methods have been proposed for deep model compression in recent years, almost all of them have focused on reducing storage complexity. In this work, we extend the teacher-student framework for deep model com- pression, since it has the potential to address runtime and train time complexity too. We propose a simple method- ology to include a noise-based regularizer while training the student from the teacher, which provides a healthy im- provement in the performance of the student network. Our experiments on the CIFAR-10, SVHN and MNIST datasets show promising improvement, with the best performance on the CIFAR-10 dataset. We also conduct a comprehensive empirical evaluation of the proposed method under related settings on the CIFAR-10 dataset to show the promise of the proposed approach. |
| نوع الوثيقة: | article in journal/newspaper |
| وصف الملف: | text |
| اللغة: | English |
| Relation: | http://raiith.iith.ac.in/2877/1/1610.09650.pdf; Sau, B B and Balasubramanian, Vineeth N (2016) Deep Model Compression: Distilling Knowledge from Noisy Teachers. arXiv, v2. pp. 1-9. |
| الاتاحة: | http://raiith.iith.ac.in/2877/ https://arxiv.org/abs/1610.09650 http://raiith.iith.ac.in/2877/1/1610.09650.pdf |
| رقم الانضمام: | edsbas.888794F1 |
| قاعدة البيانات: | BASE |
| الوصف غير متاح. |