Document Type

Article

Publication Date

9-2025

Publication Title

Intelligent Systems with Applications

Abstract

To address the large model sizes and intensive computation requirements of deep neural networks (DNNs), weight pruning techniques have been proposed and generally fall into two categories: static regularization-based pruning and dynamic regularization-based pruning. However, the static method often leads to either complex operations or reduced accuracy, while the dynamic method requires extensive time to adjust parameters to maintain accuracy while achieving effective pruning. In this paper, we propose a unified robustness-aware framework for DNN weight pruning that dynamically updates regularization terms bounded by the designated constraint. This framework can generate both non-structured sparsity and different kinds of structured sparsity, and it incorporates adversarial training to enhance the robustness of the sparse model. We further extend our approach into an integrated framework capable of handling multiple DNN compression tasks. Experimental results show that our proposed method increases the compression rate-up to 630x for LeNet-5, 45x for AlexNet, 7.2x for MobileNet, 3.2x for ResNet-50-while also reducing training time and simplifying hyperparameter tuning to a single penalty parameter. Additionally, our method improves model robustness by 5.07% for ResNet-18 and 3.34% for VGG-16 under a 16xpruning rate, outperforming the state-of-the-art ADMM-based hard constraint method.

Comments

This research was supported by the National Science Foundation, United States under awards CAREER CMMI-1750531, ECCS-1609916, CCF-1919117, CNS-1909172, CNS-1739748, and CNS-2245765.

DOI

10.1016/j.iswa.2025.200556

Version

Publisher's PDF

Volume

27

Download

Share

COinS