Conference
Are Transformers More Robust? Towards Exact Robustness Verification for Transformers
| العنوان: | Are Transformers More Robust? Towards Exact Robustness Verification for Transformers |
|---|---|
| المؤلفون: | Liao, Brian Hsuan-Cheng, Cheng, Chih-Hong, Esen, Hasan, Knoll, Alois |
| سنة النشر: | 2023 |
| المجموعة: | Publikationsdatenbank der Fraunhofer-Gesellschaft |
| مصطلحات موضوعية: | neural networks, NN, transformer, robustness, safety, Mixed Integer Quadratically Constrained Programming, MIQCP, Multi-Layer-Perceptron, MLP, safety-critical, neural networks verification, lane departure warning, autonomous driving |
| الوصف: | 89 ; 103 ; As an emerging type of Neural Networks (NNs), Transformers are used in many domains ranging from Natural Language Processing to Autonomous Driving. In this paper, we study the robustness problem of Transformers, a key characteristic as low robustness may cause safety concerns. Specifically, we focus on Sparsemax-based Transformers and reduce the finding of their maximum robustness to a Mixed Integer Quadratically Constrained Programming (MIQCP) problem. We also design two pre-processing heuristics that can be embedded in the MIQCP encoding and substantially accelerate its solving. We then conduct experiments using the application of Land Departure Warning to compare the robustness of Sparsemax-based Transformers against that of the more conventional Multi-Layer-Perceptron (MLP) NNs. To our surprise, Transformers are not necessarily more robust, leading to profound considerations in selecting appropriate NN architectures for safety-critical domain applications. |
| نوع الوثيقة: | conference object |
| اللغة: | English |
| ردمك: | 978-3-031-40923-3 3-031-40923-X |
| Relation: | International Conference on Computer Safety, Reliability and Security 2023; Computer Safety, Reliability, and Security. 42nd International Conference, SAFECOMP 2023. Proceedings; FOUNDATIONS FOR CONTINUOUS ENGINEERING OF TRUSTWORTHY AUTONOMY; 956123; https://publica.fraunhofer.de/handle/publica/451207 |
| DOI: | 10.1007/978-3-031-40923-3_8 |
| الاتاحة: | https://publica.fraunhofer.de/handle/publica/451207 https://doi.org/10.1007/978-3-031-40923-3_8 |
| رقم الانضمام: | edsbas.1091BBBA |
| قاعدة البيانات: | BASE |
| ردمك: | 9783031409233 303140923X |
|---|---|
| DOI: | 10.1007/978-3-031-40923-3_8 |