Modern compression and acceleration methods for exploring efficient deep neural networks render real-world applications more feasible.Existing approaches uniformly apply the same procedure to every input image,overloo...Modern compression and acceleration methods for exploring efficient deep neural networks render real-world applications more feasible.Existing approaches uniformly apply the same procedure to every input image,overlooking instancewise complexity variations.Moreover,owing to pruning or decomposition techniques,the upper bound of network representation capabilities might be permanently diminished.In this work,an input-dependent multiscale dynamic inference method(MSDI)is developed to strike a better balance between model performance and inference acceleration.Specifically,we modify the main body of a convolutional network to obtain a series of parameter-sharing subnetworks with varying levels of complexity.A side branch structure is then introduced to assign an input instance to a suitable subnetwork as its inference route,and we expect to accelerate the inference by assigning the easy input to the subnetwork with low capacity.We further propose multiscale distillation training to optimize the training of the modified subnetworks.Additionally,we compare the entropy-based and learning-based grading approaches,aiming to obtain a more suitable route assignment method.Experiments show that MSDI can accelerate most existing convolutional models,achieving up to 74.7%computation savings across diverse datasets.展开更多
文摘Modern compression and acceleration methods for exploring efficient deep neural networks render real-world applications more feasible.Existing approaches uniformly apply the same procedure to every input image,overlooking instancewise complexity variations.Moreover,owing to pruning or decomposition techniques,the upper bound of network representation capabilities might be permanently diminished.In this work,an input-dependent multiscale dynamic inference method(MSDI)is developed to strike a better balance between model performance and inference acceleration.Specifically,we modify the main body of a convolutional network to obtain a series of parameter-sharing subnetworks with varying levels of complexity.A side branch structure is then introduced to assign an input instance to a suitable subnetwork as its inference route,and we expect to accelerate the inference by assigning the easy input to the subnetwork with low capacity.We further propose multiscale distillation training to optimize the training of the modified subnetworks.Additionally,we compare the entropy-based and learning-based grading approaches,aiming to obtain a more suitable route assignment method.Experiments show that MSDI can accelerate most existing convolutional models,achieving up to 74.7%computation savings across diverse datasets.
