Academic Journal
Filter pruning for convolutional neural networks in semantic image segmentation
| العنوان: | Filter pruning for convolutional neural networks in semantic image segmentation |
|---|---|
| المؤلفون: | López González, Clara Isabel, Gascó, Esther, Barrientos Espillco, Fredy, Besada Portas, Eva, Pajares Martínsanz, Gonzalo |
| بيانات النشر: | Elsevier |
| سنة النشر: | 2024 |
| المجموعة: | Universidad Complutense de Madrid (UCM): E-Prints Complutense |
| مصطلحات موضوعية: | 004.8, 004.85, 004.932, 004.032.26, Convolutional Neural Networks (CNNs), Explainable Artificial Intelligence (xAI), Filter pruning, Image segmentation, Model compression, Principal Component Analysis (PCA), Inteligencia artificial (Informática), 1203.17 Informática |
| الوصف: | The remarkable performance of Convolutional Neural Networks (CNNs) has increased their use in real-time systems and devices with limited resources. Hence, compacting these networks while preserving accuracy has become necessary, leading to multiple compression methods. However, the majority require intensive iterative procedures and do not delve into the influence of the used data. To overcome these issues, this paper presents several contributions, framed in the context of explainable Artificial Intelligence (xAI): (a) two filter pruning methods for CNNs, which remove the less significant convolutional kernels; (b) a fine-tuning strategy to recover generalization; (c) a layer pruning approach for U-Net; and (d) an explanation of the relationship between performance and the used data. Filter and feature maps information are used in the pruning process: Principal Component Analysis (PCA) is combined with a next-convolution influence-metric, while the latter and the mean standard deviation are used in an importance score distribution-based method. The developed strategies are generic, and therefore applicable to different models. Experiments demonstrating their effectiveness are conducted over distinct CNNs and datasets, focusing mainly on semantic segmentation (using U-Net, DeepLabv3+, SegNet, and VGG-16 as highly representative models). Pruned U-Net on agricultural benchmarks achieves 98.7% parameters and 97.5% FLOPs drop, with a 0.35% gain in accuracy. DeepLabv3+ and SegNet on CamVid reach 46.5% and 72.4% parameters reduction and a 51.9% and 83.6% FLOPs drop respectively, with almost no decrease in accuracy. VGG-16 on CIFAR-10 obtains up to 86.5% parameter and 82.2% FLOPs decrease with a 0.78% accuracy gain. ; Depto. de Ingeniería de Software e Inteligencia Artificial (ISIA) ; Sección Deptal. de Arquitectura de Computadores y Automática (Físicas) ; Fac. de Informática ; Fac. de Ciencias Físicas ; TRUE ; Sinérgicos Comunidad de Madrid ; Ministerio de Ciencia, Innovación y Universidades de España ; Ministerio de ... |
| نوع الوثيقة: | article in journal/newspaper |
| وصف الملف: | application/pdf |
| اللغة: | English |
| تدمد: | 0893-6080 |
| Relation: | info:eu-repo/grantAgreement/EC/H2020/954755/EU/Hacia un sistema Integral para la Alerta y Gestión de BLOOMs de cianobacterias en aguas continentales/IA-GES-BLOOM-CM; info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/RTI2018-098962-B-C21/ES/MONITORIZACION AUTOMATICA DE CONTAMINANTES EN AGUAS EMBALSADAS UTILIZANDO BIOSENSORES Y VEHICULOS AUTONOMOS DE SUPERFICIE/; López-González CI, Gascó E, Barrientos-Espillco F, Besada-Portas E, Pajares G. Filter pruning for convolutional neural networks in semantic image segmentation. Neural Networks. 2024 Jan;169:713-32; https://www.sciencedirect.com/science/article/pii/S0893608023006330; https://hdl.handle.net/20.500.14352/88802 |
| DOI: | 10.1016/j.neunet.2023.11.010 |
| الاتاحة: | https://hdl.handle.net/20.500.14352/88802 https://doi.org/10.1016/j.neunet.2023.11.010 https://www.sciencedirect.com/science/article/pii/S0893608023006330 |
| Rights: | Attribution 4.0 International ; open access ; http://creativecommons.org/licenses/by/4.0/ |
| رقم الانضمام: | edsbas.962FD319 |
| قاعدة البيانات: | BASE |
| ResultId |
1 |
|---|---|
| Header |
edsbas BASE edsbas.962FD319 995 3 Academic Journal academicJournal 995.377258300781 |
| PLink |
https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsbas&AN=edsbas.962FD319&custid=s6537998&authtype=sso |
| FullText |
Array
(
[Availability] => 0
)
Array ( [0] => Array ( [Url] => https://hdl.handle.net/20.500.14352/88802# [Name] => EDS - BASE [Category] => fullText [Text] => View record in BASE [MouseOverText] => View record in BASE ) ) |
| Items |
Array
(
[Name] => Title
[Label] => Title
[Group] => Ti
[Data] => Filter pruning for convolutional neural networks in semantic image segmentation
)
Array ( [Name] => Author [Label] => Authors [Group] => Au [Data] => <searchLink fieldCode="AR" term="%22López+González%2C+Clara+Isabel%22">López González, Clara Isabel</searchLink><br /><searchLink fieldCode="AR" term="%22Gascó%2C+Esther%22">Gascó, Esther</searchLink><br /><searchLink fieldCode="AR" term="%22Barrientos+Espillco%2C+Fredy%22">Barrientos Espillco, Fredy</searchLink><br /><searchLink fieldCode="AR" term="%22Besada+Portas%2C+Eva%22">Besada Portas, Eva</searchLink><br /><searchLink fieldCode="AR" term="%22Pajares+Martínsanz%2C+Gonzalo%22">Pajares Martínsanz, Gonzalo</searchLink> ) Array ( [Name] => Publisher [Label] => Publisher Information [Group] => PubInfo [Data] => Elsevier ) Array ( [Name] => DatePubCY [Label] => Publication Year [Group] => Date [Data] => 2024 ) Array ( [Name] => Subset [Label] => Collection [Group] => HoldingsInfo [Data] => Universidad Complutense de Madrid (UCM): E-Prints Complutense ) Array ( [Name] => Subject [Label] => Subject Terms [Group] => Su [Data] => <searchLink fieldCode="DE" term="%22004%2E8%22">004.8</searchLink><br /><searchLink fieldCode="DE" term="%22004%2E85%22">004.85</searchLink><br /><searchLink fieldCode="DE" term="%22004%2E932%22">004.932</searchLink><br /><searchLink fieldCode="DE" term="%22004%2E032%2E26%22">004.032.26</searchLink><br /><searchLink fieldCode="DE" term="%22Convolutional+Neural+Networks+%28CNNs%29%22">Convolutional Neural Networks (CNNs)</searchLink><br /><searchLink fieldCode="DE" term="%22Explainable+Artificial+Intelligence+%28xAI%29%22">Explainable Artificial Intelligence (xAI)</searchLink><br /><searchLink fieldCode="DE" term="%22Filter+pruning%22">Filter pruning</searchLink><br /><searchLink fieldCode="DE" term="%22Image+segmentation%22">Image segmentation</searchLink><br /><searchLink fieldCode="DE" term="%22Model+compression%22">Model compression</searchLink><br /><searchLink fieldCode="DE" term="%22Principal+Component+Analysis+%28PCA%29%22">Principal Component Analysis (PCA)</searchLink><br /><searchLink fieldCode="DE" term="%22Inteligencia+artificial+%28Informática%29%22">Inteligencia artificial (Informática)</searchLink><br /><searchLink fieldCode="DE" term="%221203%2E17+Informática%22">1203.17 Informática</searchLink> ) Array ( [Name] => Abstract [Label] => Description [Group] => Ab [Data] => The remarkable performance of Convolutional Neural Networks (CNNs) has increased their use in real-time systems and devices with limited resources. Hence, compacting these networks while preserving accuracy has become necessary, leading to multiple compression methods. However, the majority require intensive iterative procedures and do not delve into the influence of the used data. To overcome these issues, this paper presents several contributions, framed in the context of explainable Artificial Intelligence (xAI): (a) two filter pruning methods for CNNs, which remove the less significant convolutional kernels; (b) a fine-tuning strategy to recover generalization; (c) a layer pruning approach for U-Net; and (d) an explanation of the relationship between performance and the used data. Filter and feature maps information are used in the pruning process: Principal Component Analysis (PCA) is combined with a next-convolution influence-metric, while the latter and the mean standard deviation are used in an importance score distribution-based method. The developed strategies are generic, and therefore applicable to different models. Experiments demonstrating their effectiveness are conducted over distinct CNNs and datasets, focusing mainly on semantic segmentation (using U-Net, DeepLabv3+, SegNet, and VGG-16 as highly representative models). Pruned U-Net on agricultural benchmarks achieves 98.7% parameters and 97.5% FLOPs drop, with a 0.35% gain in accuracy. DeepLabv3+ and SegNet on CamVid reach 46.5% and 72.4% parameters reduction and a 51.9% and 83.6% FLOPs drop respectively, with almost no decrease in accuracy. VGG-16 on CIFAR-10 obtains up to 86.5% parameter and 82.2% FLOPs decrease with a 0.78% accuracy gain. ; Depto. de Ingeniería de Software e Inteligencia Artificial (ISIA) ; Sección Deptal. de Arquitectura de Computadores y Automática (Físicas) ; Fac. de Informática ; Fac. de Ciencias Físicas ; TRUE ; Sinérgicos Comunidad de Madrid ; Ministerio de Ciencia, Innovación y Universidades de España ; Ministerio de ... ) Array ( [Name] => TypeDocument [Label] => Document Type [Group] => TypDoc [Data] => article in journal/newspaper ) Array ( [Name] => Format [Label] => File Description [Group] => SrcInfo [Data] => application/pdf ) Array ( [Name] => Language [Label] => Language [Group] => Lang [Data] => English ) Array ( [Name] => ISSN [Label] => ISSN [Group] => ISSN [Data] => 0893-6080 ) Array ( [Name] => NoteTitleSource [Label] => Relation [Group] => SrcInfo [Data] => info:eu-repo/grantAgreement/EC/H2020/954755/EU/Hacia un sistema Integral para la Alerta y Gestión de BLOOMs de cianobacterias en aguas continentales/IA-GES-BLOOM-CM; info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/RTI2018-098962-B-C21/ES/MONITORIZACION AUTOMATICA DE CONTAMINANTES EN AGUAS EMBALSADAS UTILIZANDO BIOSENSORES Y VEHICULOS AUTONOMOS DE SUPERFICIE/; López-González CI, Gascó E, Barrientos-Espillco F, Besada-Portas E, Pajares G. Filter pruning for convolutional neural networks in semantic image segmentation. Neural Networks. 2024 Jan;169:713-32; https://www.sciencedirect.com/science/article/pii/S0893608023006330; https://hdl.handle.net/20.500.14352/88802 ) Array ( [Name] => DOI [Label] => DOI [Group] => ID [Data] => 10.1016/j.neunet.2023.11.010 ) Array ( [Name] => URL [Label] => Availability [Group] => URL [Data] => https://hdl.handle.net/20.500.14352/88802<br />https://doi.org/10.1016/j.neunet.2023.11.010<br />https://www.sciencedirect.com/science/article/pii/S0893608023006330 ) Array ( [Name] => Copyright [Label] => Rights [Group] => Cpyrght [Data] => Attribution 4.0 International ; open access ; http://creativecommons.org/licenses/by/4.0/ ) Array ( [Name] => AN [Label] => Accession Number [Group] => ID [Data] => edsbas.962FD319 ) |
| RecordInfo |
Array
(
[BibEntity] => Array
(
[Identifiers] => Array
(
[0] => Array
(
[Type] => doi
[Value] => 10.1016/j.neunet.2023.11.010
)
)
[Languages] => Array
(
[0] => Array
(
[Text] => English
)
)
[Subjects] => Array
(
[0] => Array
(
[SubjectFull] => 004.8
[Type] => general
)
[1] => Array
(
[SubjectFull] => 004.85
[Type] => general
)
[2] => Array
(
[SubjectFull] => 004.932
[Type] => general
)
[3] => Array
(
[SubjectFull] => 004.032.26
[Type] => general
)
[4] => Array
(
[SubjectFull] => Convolutional Neural Networks (CNNs)
[Type] => general
)
[5] => Array
(
[SubjectFull] => Explainable Artificial Intelligence (xAI)
[Type] => general
)
[6] => Array
(
[SubjectFull] => Filter pruning
[Type] => general
)
[7] => Array
(
[SubjectFull] => Image segmentation
[Type] => general
)
[8] => Array
(
[SubjectFull] => Model compression
[Type] => general
)
[9] => Array
(
[SubjectFull] => Principal Component Analysis (PCA)
[Type] => general
)
[10] => Array
(
[SubjectFull] => Inteligencia artificial (Informática)
[Type] => general
)
[11] => Array
(
[SubjectFull] => 1203.17 Informática
[Type] => general
)
)
[Titles] => Array
(
[0] => Array
(
[TitleFull] => Filter pruning for convolutional neural networks in semantic image segmentation
[Type] => main
)
)
)
[BibRelationships] => Array
(
[HasContributorRelationships] => Array
(
[0] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => López González, Clara Isabel
)
)
)
[1] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Gascó, Esther
)
)
)
[2] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Barrientos Espillco, Fredy
)
)
)
[3] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Besada Portas, Eva
)
)
)
[4] => Array
(
[PersonEntity] => Array
(
[Name] => Array
(
[NameFull] => Pajares Martínsanz, Gonzalo
)
)
)
)
[IsPartOfRelationships] => Array
(
[0] => Array
(
[BibEntity] => Array
(
[Dates] => Array
(
[0] => Array
(
[D] => 01
[M] => 01
[Type] => published
[Y] => 2024
)
)
[Identifiers] => Array
(
[0] => Array
(
[Type] => issn-print
[Value] => 08936080
)
[1] => Array
(
[Type] => issn-locals
[Value] => edsbas
)
[2] => Array
(
[Type] => issn-locals
[Value] => edsbas.oa
)
)
)
)
)
)
)
|
| IllustrationInfo |