Report
Masked Autoencoders are Efficient Class Incremental Learners
| العنوان: | Masked Autoencoders are Efficient Class Incremental Learners |
|---|---|
| المؤلفون: | Zhai, Jiang-Tian, Liu, Xialei, Bagdanov, Andrew D., Li, Ke, Cheng, Ming-Ming |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Machine Learning |
| الوصف: | Class Incremental Learning (CIL) aims to sequentially learn new classes while avoiding catastrophic forgetting of previous knowledge. We propose to use Masked Autoencoders (MAEs) as efficient learners for CIL. MAEs were originally designed to learn useful representations through reconstructive unsupervised learning, and they can be easily integrated with a supervised loss for classification. Moreover, MAEs can reliably reconstruct original input images from randomly selected patches, which we use to store exemplars from past tasks more efficiently for CIL. We also propose a bilateral MAE framework to learn from image-level and embedding-level fusion, which produces better-quality reconstructed images and more stable representations. Our experiments confirm that our approach performs better than the state-of-the-art on CIFAR-100, ImageNet-Subset, and ImageNet-Full. The code is available at https://github.com/scok30/MAE-CIL . Comment: Accepted at ICCV 2023 |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2308.12510 |
| رقم الانضمام: | edsarx.2308.12510 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |