Abstract

We present a dynamic network rewiring (DNR) method to generate pruned deep neural network (DNN) models that both are robust against adversarially generated images and maintain high accuracy on clean images. In particular, the disclosed DNR training method is based on a unified constrained optimization formulation using a novel hybrid loss function that merges sparse learning with robust adversarial training. This training strategy dynamically adjusts inter-layer connectivity based on per-layer normalized momentum computed from the hybrid loss function. To further improve the robustness of the pruned models, we propose DNR++, an extension of the DNR method where we introduce the idea of sparse parametric Gaussian noise tensor that is added to the weight tensors to yield robust regularization. In contrast to existing robust pruning frameworks that require multiple training iterations, the proposed …

Metadata

publication

ACM Transactions on Embedded Computing Systems 21 (5), 1-24, 2022

year

2022

publication date

2022/12/13

authors

Souvik Kundu, Yao Fu, Bill Ye, Peter A Beerel, Massoud Pedram

link

https://dl.acm.org/doi/abs/10.1145/3510833

resource_link

https://dl.acm.org/doi/pdf/10.1145/3510833

journal

ACM Transactions on Embedded Computing Systems

volume

21

issue

5

pages

1-24

publisher

ACM