-
1Dissertation/ Thesis
المؤلفون: 黃名儀, Huang, Ming-I
المساهمون: 郁方 洪智鐸, Yu, Fang Hong, Chih-Duo
مصطلحات موضوعية: 政治大學, 深度神經網路, 動態符號執行測試, 公平性測試, NCCU, Concolic Testing, Fairness Testing, Deep Neural Networks
وصف الملف: 1636132 bytes; application/pdf
Relation: Aggarwal, A., Lohia, P., Nagar, S., Dey, K., and Saha, D. (2019). Black box fairness testing of machine learning models. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 625–635. Albarghouthi, A., D’Antoni, L., Drews, S., and Nori, A. V. (2017). Fairsquare: prob- abilistic verification of program fairness. Proceedings of the ACM on Programming Languages, 1(OOPSLA):1–30. Awwad, Y., Fletcher, R., Frey, D., Gandhi, A., Najafian, M., and Teodorescu, M. (2020). Exploring fairness in machine learning for international development. Technical report, CITE MIT D-Lab. Bastani, O., Zhang, X., and Solar-Lezama, A. (2019). Probabilistic verification of fairness properties via concentration. Proceedings of the ACM on Programming Languages, 3(OOPSLA):1–27. Biswas, S. and Rajan, H. (2020). Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pages 642–653. Biswas, S. and Rajan, H. (2021). Fair preprocessing: towards understanding compositional fairness of data transformers in machine learning pipeline. In Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pages 981–993. Biswas, S. and Rajan, H. (2023). Fairify: Fairness verification of neural networks. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1546–1558. IEEE. Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., et al. (2016). End to end learning for self- driving cars. arXiv preprint arXiv:1604.07316. Buolamwini, J. and Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR. Chakraborty, J., Majumder, S., Yu, Z., and Menzies, T. (2020). Fairway: a way to build fair ml software. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 654–665. Chen, Y.-F., Tsai, W.-L., Wu, W.-C., Yen, D.-D., and Yu, F. (2021). Pyct: A python con- colic tester. In Programming Languages and Systems: 19th Asian Symposium, APLAS 2021, Chicago, IL, USA, October 17–18, 2021, Proceedings 19, pages 38–46. Springer. Chen, Z., Zhang, J. M., Sarro, F., and Harman, M. (2024). Fairness improvement with multiple protected attributes: How far are we? In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1–13. Dastin, J. (2022). Amazon scraps secret ai recruiting tool that showed bias against women. In Ethics of data and analytics, pages 296–299. Auerbach Publications. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science con- ference, pages 214–226. Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 259–268. Flores, A. W., Bechtel, K., and Lowenkamp, C. T. (2016). False positives, false nega- tives, and false analyses: A rejoinder to machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. Fed. Probation, 80:38. Galhotra, S., Brun, Y., and Meliou, A. (2017). Fairness testing: testing software for dis- crimination. In Proceedings of the 2017 11th Joint meeting on foundations of software engineering, pages 498–510. Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., and Vechev, M. (2018). Ai2: Safety and robustness certification of neural networks with abstract inter- pretation. In 2018 IEEE symposium on security and privacy (SP), pages 3–18. IEEE. Gohar, U., Biswas, S., and Rajan, H. (2023). Towards understanding fairness and its composition in ensemble machine learning. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1533–1545. IEEE. Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adver- sarial examples. arXiv preprint arXiv:1412.6572. Hort, M., Zhang, J. M., Sarro, F., and Harman, M. (2021). Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 994–1006. John, P. G., Vijaykeerthy, D., and Saha, D. (2020). Verifying individual fairness in machine learning models. In Conference on Uncertainty in Artificial Intelligence, pages 749– 758. PMLR. Katz, G., Barrett, C., Dill, D. L., Julian, K., and Kochenderfer, M. J. (2017). Reluplex: An efficient smt solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30, pages 97–117. Springer. Katz, G., Huang, D. A., Ibeling, D., Julian, K., Lazarus, C., Lim, R., Shah, P., Thakoor, S., Wu, H., Zeljić, A., et al. (2019). The marabou framework for verification and analysis of deep neural networks. In Computer Aided Verification: 31st International Conference, CAV 2019, New York City, NY, USA, July 15-18, 2019, Proceedings, Part I 31, pages 443–452. Springer. Khedr, H. and Shoukry, Y. (2023). Certifair: A framework for certified global fairness of neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8237–8245. Kurakin, A., Goodfellow, I. J., and Bengio, S. (2018). Adversarial examples in the phys- ical world. In Artificial intelligence safety and security, pages 99–112. Chapman and Hall/CRC. Lam, L. and Suen, S. (1997). Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 27(5):553–568. Li, T., Xie, X., Wang, J., Guo, Q., Liu, A., Ma, L., and Liu, Y. (2023). Faire: Repair- ing fairness of neural networks via neuron condition synthesis. ACM Transactions on Software Engineering and Methodology, 33(1):1–24. Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., Van Der Laak, J. A., Van Ginneken, B., and Sánchez, C. I. (2017). A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88. Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., Chen, C., Su, T., Li, L., Liu, Y., et al. (2018). Deepgauge: Multi-granularity testing criteria for deep learning systems. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pages 120–131. Mohammadi, K., Sivaraman, A., and Farnadi, G. (2023). Feta: Fairness enforced verify- ing, training, and predicting algorithms for neural networks. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1–11. Pei, K., Cao, Y., Yang, J., and Jana, S. (2017). Deepxplore: Automated whitebox testing of deep learning systems. In proceedings of the 26th Symposium on Operating Systems Principles, pages 1–18. Ruoss, A., Balunovic, M., Fischer, M., and Vechev, M. (2020). Learning certified in- dividually fair representations. Advances in neural information processing systems, 33:7584–7596. Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., and Ghani, R. (2018). Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577. Sharma, A. and Wehrheim, H. (2020). Automatic fairness testing of machine learning models. In Testing Software and Systems: 32nd IFIP WG 6.1 International Conference, ICTSS 2020, Naples, Italy, December 9–11, 2020, Proceedings 32, pages 255–271. Springer. Singh, G., Gehr, T., Püschel, M., and Vechev, M. (2019). An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages, 3(POPL):1–30. Sun, Y., Wu, M., Ruan, W., Huang, X., Kwiatkowska, M., and Kroening, D. (2018). Concolic testing for deep neural networks. In Proceedings of the 33rd ACM/IEEE In- ternational Conference on Automated Software Engineering, pages 109–119. Udeshi, S., Arora, P., and Chattopadhyay, S. (2018). Automated directed fairness test- ing. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pages 98–108. Urban, C., Christakis, M., Wüstholz, V., and Zhang, F. (2020). Perfectly parallel fairness certification of neural networks. Proceedings of the ACM on Programming Languages, 4(OOPSLA):1–30. Wang, S., Pei, K., Whitehouse, J., Yang, J., and Jana, S. (2018). Formal security analysis of neural networks using symbolic intervals. In 27th USENIX Security Symposium (USENIX Security 18), pages 1599–1614. Yurochkin, M., Bower, A., and Sun, Y. (2019). Training individually fair ml models with sensitive subspace robustness. arXiv preprint arXiv:1907.00020. Zhang, J. M. and Harman, M. (2021). “ignorance and prejudice” in software fairness. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 1436–1447. IEEE. Zhang, L., Zhang, Y., and Zhang, M. (2021). Efficient white-box fairness testing through gradient search. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 103–114. Zhang, P., Wang, J., Sun, J., Dong, G., Wang, X., Wang, X., Dong, J. S., and Dai, T. (2020). White-box fairness testing through adversarial sampling. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 949–960. Zheng, H., Chen, Z., Du, T., Zhang, X., Cheng, Y., Ji, S., Wang, J., Yu, Y., and Chen, J. (2022). Neuronfair: Interpretable white-box fairness testing through biased neuron identification. In 44th International Conference on Software Engineering, pages 1–13, New York, NY, USA. ACM.; G0111356047; https://nccur.lib.nccu.edu.tw//handle/140.119/153165; https://nccur.lib.nccu.edu.tw/bitstream/140.119/153165/1/604701.pdf
-
2Dissertation/ Thesis
المؤلفون: 蔣其叡, Chiang, Chi-Rui
المساهمون: 郁方, Yu, Fang
مصطلحات موضوعية: 動態符號執行測試, 自動單元測試, Automatic Unit Testing, Concolic Testing, Python, Dynamic Concolication, NCCU
وصف الملف: 752779 bytes; application/pdf
Relation: Ahmadilivani, M. H., Taheri, M., Raik, J., Daneshtalab, M., and Jenihhin, M. (2023). A systematic literature review on hardware reliability assessment methods for deep neural networks. Araki., L. Y. and Peres., L. M. (2018). A systematic review of concolic testing with aplication of test criteria. In Proceedings of the 20th International Conference on Enterprise Information Systems - Volume 2: ICEIS, pages 121–132. INSTICC, SciTePress. Bai, T., Huang, S., Huang, Y., Wang, X., Xia, C., Qu, Y., and Yang, Z. (2024). Criticalfuzz: A critical neuron coverage-guided fuzz testing framework for deep neural networks. Information and Software Technology, 172:107476. Ball, T. and Daniel, J. (2015). Deconstructing dynamic symbolic execution. In Irlbeck, M., Peled, D. A., and Pretschner, A., editors, Dependable Software Systems Engineering, volume 40 of NATO Science for Peace and Security Series, D: Information and Communication Security, pages 26–41. IOS Press. Cadar, C. and Sen, K. (2013). Symbolic execution for software testing: three decades later. Commun. ACM, 56(2):82–90. Caniço, A. B. and Santos, A. L. (2023). Witter: A library for white-box testing of introductory programming algorithms. In Proceedings of the 2023 ACM SIGPLAN Interna-tional Symposium on SPLASH-E, SPLASH-E 2023, page 69–74, New York, NY, USA. Association for Computing Machinery. Chen, Y.-F., Tsai, W.-L., Wu, W.-C., Yen, D.-D., and Yu, F. (2021). Pyct: A python concolic tester. In Oh, H., editor, Programming Languages and Systems, pages 38–46, Cham. Springer International Publishing. Gopinath, D., Wang, K., Zhang, M., Pasareanu, C. S., and Khurshid, S. (2018). Symbolic execution for deep neural networks. Gu, J., Luo, X., Zhou, Y., and Wang, X. (2022). Muffin: Testing deep learning libraries via neural architecture fuzzing. Huang, J.-t., Zhang, J., Wang, W., He, P., Su, Y., and Lyu, M. R. (2022). Aeon: A method for automatic evaluation of nlp test cases. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2022, page 202– 214, New York, NY, USA. Association for Computing Machinery. Ji, P., Feng, Y., Liu, J., Zhao, Z., and Chen, Z. (2022). Asrtest: Automated testing for deepneural- network-driven speech recognition systems. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2022, page 189–201, New York, NY, USA. Association for Computing Machinery. Khan, M. (2011). Different approaches to black box testing technique for finding errors. International Journal of Software Engineering Applications, 2. Klees, G., Ruef, A., Cooper, B., Wei, S., and Hicks, M. (2018). Evaluating fuzz testing. Li, R., Yang, P., Huang, C.-C., Sun, Y., Xue, B., and Zhang, L. (2022). Towards practical robustness analysis for dnns based on pac-model learning. In Proceedings of the 44th International Conference on Software Engineering, ICSE ’22, page 2189–2201, New York, NY, USA. Association for Computing Machinery. Liu, Z., Feng, Y., and Chen, Z. (2021). Dialtest: Automated testing for recurrent-neuralnetwork- driven dialogue systems. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2021, page 115–126, New York, NY, USA. Association for Computing Machinery. Manès, V. J., Han, H., Han, C., Cha, S. K., Egele, M., Schwartz, E. J., and Woo, M. (2021). The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering, 47(11):2312–2331. Sen, K., Marinov, D., and Agha, G. (2005). Cute: a concolic unit testing engine for c. In Proceedings of the 10th European Software Engineering Conference Held Jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-13, page 263–272, New York, NY, USA. Association for Computing Machinery. Wang, S., Shrestha, N., Subburaman, A. K., Wang, J., Wei, M., and Nagappan, N. (2021a). Automatic unit test generation for machine learning libraries: How far are we? In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 1548–1560. Wang, Z., You, H., Chen, J., Zhang, Y., Dong, X., and Zhang, W. (2021b). Prioritizing test inputs for deep neural networks via mutation analysis. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 397–409. Xia, C. S., Dutta, S., Misailovic, S., Marinov, D., and Zhang, L. (2023). Balancing effectiveness and flakiness of non-deterministic machine learning tests. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1801–1813. Xie, D., Li, Y., Kim, M., Pham, H. V., Tan, L., Zhang, X., and Godfrey, M. W. (2022). Docter: Documentation-guided fuzzing for testing deep learning api functions. ISSTA 2022, page 176–188, New York, NY, USA. Association for Computing Machinery. Yang, C., Deng, Y., Yao, J., Tu, Y., Li, H., and Zhang, L. (2023). Fuzzing automatic differentiation in deep-learning libraries. Yu, F., Chi, Y.-Y., and Chen, Y.-F. (2024a). Constraint-based adversarial example synthesis. Yu, F., Chi, Y.-Y., and Chen, Y.-F. (2024b). Constraint-based adversarial example synthesis. Zhang, J. and Li, J. (2020). Testing and verification of neural-network-based safety-critical control software: A systematic literature review. Information and Software Technology, 123:106296. Zhang, X., Sun, N., Fang, C., Liu, J., Liu, J., Chai, D., Wang, J., and Chen, Z. (2021). Predoo: Precision testing of deep learning operators. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2021, page 400–412, New York, NY, USA. Association for Computing Machinery. Zhao, X., Qu, H., Xu, J., Li, X., Lv, W., and Wang, G.-G. (2023). A systematic review of fuzzing. Soft Comput., 28(6):5493–5522.; G0111356024; https://nccur.lib.nccu.edu.tw//handle/140.119/153152; https://nccur.lib.nccu.edu.tw/bitstream/140.119/153152/1/602401.pdf
-
3Dissertation/ Thesis
المؤلفون: 紀亞妤, Chi, Ya-Yu
المساهمون: 郁方, Yu, Fang
مصطلحات موضوعية: 動態符號執行測試, 對抗式生成攻擊, 神經網路模型, Concolic Testing, Adversarial attack, Neural Network model
وصف الملف: 4467534 bytes; application/pdf
Relation: [1] Pyct-rq: Constraint-based concolic testing for neural networks. https://github.com/ManticoreDai/PyCT-rq. [2] S. Cha, S. Hong, J. Bak, J. Kim, J. Lee, and H. Oh. Enhancing dynamic symbolic execution by automatically learning search heuristics. IEEE Transactions on Software Engineering, 48(9):3640–3663, 2021. [3] H. Chen, S. M. Lundberg, and S.-I. Lee. Explaining a series of models by propagating shapley values. Nature communications, 13(1):4512, 2022. [4] Y.-F. Chen, W.-L. Tsai, W.-C. Wu, D.-D. Yen, and F. Yu. Pyct: A python concolic tester. In Programming Languages and Systems: 19th Asian Symposium, APLAS 2021, Chicago, IL, USA, October 17–18, 2021, Proceedings 19, pages 38–46. Springer, 2021. [5] S. Fortz, F. Mesnard, E. Payet, G. Perrouin, W. Vanhoof, and G. Vidal. An smt-based concolic testing tool for logic programs. In International Symposium on Functional and Logic Programming, pages 215–219. Springer, 2020. [6] I. Fursov, A. Zaytsev, N. Kluchnikov, A. Kravchenko, and E. Burnaev. Gradientbased adversarial attacks on categorical sequence models via traversing an embedded world. In Analysis of Images, Social Networks and Texts: 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15–16, 2020, Revised Selected Papers 9, pages 356–368. Springer, 2021. [7] I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. MIT press, 2016. [8] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. [9] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6):82–97, 2012. [10] W. Huang, Y. Sun, X. Zhao, J. Sharp, W. Ruan, J. Meng, and X. Huang. Coverageguided testing for recurrent neural networks. IEEE Transactions on Reliability, 71(3):1191–1206, 2021. [11] G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I 30, pages 97–117. Springer, 2017. [12] G. Katz, D. A. Huang, D. Ibeling, K. Julian, C. Lazarus, R. Lim, P. Shah, S. Thakoor, H. Wu, A. Zeljić, et al. The marabou framework for verification and analysis of deep neural networks. In Computer Aided Verification: 31st International Conference, CAV 2019, New York City, NY, USA, July 15-18, 2019, Proceedings, Part I 31, pages 443–452. Springer, 2019. [13] Y. Kim, S. Hong, and M. Kim. Target-driven compositional concolic testing with function summary refinement for effective bug detection. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 16–26, 2019. [14] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012. [15] Z. Li, X. Ma, C. Xu, and C. Cao. Structural coverage criteria for neural networks could be misleading. In 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pages 89–92. IEEE, 2019. [16] L. Ma, F. Juefei-Xu, F. Zhang, J. Sun, M. Xue, B. Li, C. Chen, T. Su, L. Li, Y. Liu, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering, pages 120–131, 2018. [17] L. Ma, F. Zhang, J. Sun, M. Xue, B. Li, F. Juefei-Xu, C. Xie, L. Li, Y. Liu, J. Zhao, et al. Deepmutation: Mutation testing of deep learning systems. In 2018 IEEE 29th international symposium on software reliability engineering (ISSRE), pages 100–111. IEEE, 2018. [18] X. Meng, S. Kundu, A. K. Kanuparthi, and K. Basu. Rtl-contest: Concolic testing on rtl for detecting security vulnerabilities. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 41(3):466–477, 2021. [19] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1765–1773, 2017. [20] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574–2582, 2016. [21] A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 427–436, 2015. [22] K. Pei, Y. Cao, J. Yang, and S. Jana. Deepxplore: Automated whitebox testing of deep learning systems. In proceedings of the 26th Symposium on Operating Systems Principles, pages 1–18, 2017. [23] K. Sen. Concolic testing. In Proceedings of the 22nd IEEE/ACM international conference on Automated software engineering, pages 571–572, 2007. [24] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016. [25] Y. Sun, X. Huang, D. Kroening, J. Sharp, M. Hill, and R. Ashmore. Deepconcolic: Testing and debugging deep neural networks. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pages 111–114. IEEE, 2019. [26] Y. Sun, M. Wu, W. Ruan, X. Huang, M. Kwiatkowska, and D. Kroening. Concolic testing for deep neural networks. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, pages 109–119, 2018. [27] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. [28] X. Xie, T. Li, J. Wang, L. Ma, Q. Guo, F. Juefei-Xu, and Y. Liu. Npc: N euron p ath c overage via characterizing decision logic of deep neural networks. ACM Transactions on Software Engineering and Methodology (TOSEM), 31(3):1–27, 2022. [29] X. Xie, L. Ma, F. Juefei-Xu, M. Xue, H. Chen, Y. Liu, J. Zhao, B. Li, J. Yin, and S. See. Deephunter: a coverage-guided fuzz testing framework for deep neural networks. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 146–157, 2019. [30] X. Xu, J. Chen, J. Xiao, Z. Wang, Y. Yang, and H. T. Shen. Learning optimization-based adversarial perturbations for attacking sequential recognition models. In Proceedings of the 28th ACM international conference on multimedia, pages 2802–2822, 2020. [31] Z. Zhou, W. Dou, J. Liu, C. Zhang, J. Wei, and D. Ye. Deepcon: Contribution coverage testing for deep learning systems. In 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 189–200. IEEE, 2021.; G0110356043; https://nccur.lib.nccu.edu.tw//handle/140.119/149471; https://nccur.lib.nccu.edu.tw/bitstream/140.119/149471/1/604301.pdf