FTBME: feature transferring based multi-model ensemble

التفاصيل البيبلوغرافية
العنوان: FTBME: feature transferring based multi-model ensemble
المؤلفون: B. Haijun Lv, A. Yongquan Yang, E. Zhongxi Zheng, D. Yang Wu, C. Ning Chen
المصدر: Multimedia Tools and Applications. 79:18767-18799
بيانات النشر: Springer Science and Business Media LLC, 2020.
سنة النشر: 2020
مصطلحات موضوعية: Artificial neural network, Computer Networks and Communications, Generalization, business.industry, Computer science, Property (programming), 020207 software engineering, 02 engineering and technology, Filter (signal processing), Machine learning, computer.software_genre, Field (computer science), ComputingMethodologies_PATTERNRECOGNITION, Hardware and Architecture, 0202 electrical engineering, electronic engineering, information engineering, Media Technology, Feature (machine learning), Artificial intelligence, Applications of artificial intelligence, business, computer, Software
الوصف: Multi-model ensemble is an important fundamental technique of practical value for many artificial intelligence applications. However, the usage for multi-model ensemble has been limited when it is combined with deep neural networks to construct ensemble of deep neural networks. Due to the big time and computing resources required to train and to integrate multiple deep neural networks for the achievement of multi-model ensemble, the engineering application field where developing time and computing resources are usually restricted, has not yet widespreadly benefited from ensemble of deep neural networks. To alleviate this situation, we present a new multi-model ensemble approach entitled feature transferring based multi-model ensemble (FTBME), for ensemble of deep neural networks. Primarily, we propose a feature transferring based multi-model training strategy to more affordably find multiple extra models based on a given previously optimized deep neural network model. Sequentially, to develop better ensemble solutions, we design a more effective random greedy based ensemble selection strategy to filter out models non-positive to ensemble generalization. Finally, inspired by the idea of averaging parameter points, we propose to fuse the obtained models in weight space which eventually reduces the expense of ensemble at the testing stage to a single deep neural network model while retaining the generalization. These three advances constitute the resulting technique FTBME. We conducted extensive experiments using deep neural networks, from light weight to complex, on ImageNet, CIFAR-10 and CIFAR-100. Results show that, given a deep neural network model which has been well-optimized and reaching its limit, FTBME can obtain better generalization with minor extra training expense while maintaining the expense to a single model at ensemble testing. This promising property of FTBME make us believe that it could be leveraged to broaden the usage for ensemble of deep neural networks, alleviating the situation that the engineering application field has not yet widespreadly benefited from ensemble of deep neural networks.
تدمد: 1573-7721
1380-7501
DOI: 10.1007/s11042-020-08746-4
URL الوصول: https://explore.openaire.eu/search/publication?articleId=doi_________::e248d77354997c76a485a81ef3f2804d
https://doi.org/10.1007/s11042-020-08746-4
Rights: CLOSED
رقم الانضمام: edsair.doi...........e248d77354997c76a485a81ef3f2804d
قاعدة البيانات: OpenAIRE
الوصف
تدمد:15737721
13807501
DOI:10.1007/s11042-020-08746-4