Synaptic pruning is a crucial process in synaptic refinement,eliminating unstable synaptic connections in neural circuits.This process is triggered and regulated primarily by spontaneous neural activity and experience...Synaptic pruning is a crucial process in synaptic refinement,eliminating unstable synaptic connections in neural circuits.This process is triggered and regulated primarily by spontaneous neural activity and experience-dependent mechanisms.The pruning process involves multiple molecular signals and a series of regulatory activities governing the“eat me”and“don't eat me”states.Under physiological conditions,the interaction between glial cells and neurons results in the clearance of unnecessary synapses,maintaining normal neural circuit functionality via synaptic pruning.Alterations in genetic and environmental factors can lead to imbalanced synaptic pruning,thus promoting the occurrence and development of autism spectrum disorder,schizophrenia,Alzheimer's disease,and other neurological disorders.In this review,we investigated the molecular mechanisms responsible for synaptic pruning during neural development.We focus on how synaptic pruning can regulate neural circuits and its association with neurological disorders.Furthermore,we discuss the application of emerging optical and imaging technologies to observe synaptic structure and function,as well as their potential for clinical translation.Our aim was to enhance our understanding of synaptic pruning during neural development,including the molecular basis underlying the regulation of synaptic function and the dynamic changes in synaptic density,and to investigate the potential role of these mechanisms in the pathophysiology of neurological diseases,thus providing a theoretical foundation for the treatment of neurological disorders.展开更多
The surge of large-scale models in recent years has led to breakthroughs in numerous fields,but it has also introduced higher computational costs and more complex network architectures.These increasingly large and int...The surge of large-scale models in recent years has led to breakthroughs in numerous fields,but it has also introduced higher computational costs and more complex network architectures.These increasingly large and intricate networks pose challenges for deployment and execution while also exacerbating the issue of network over-parameterization.To address this issue,various network compression techniques have been developed,such as network pruning.A typical pruning algorithm follows a three-step pipeline involving training,pruning,and retraining.Existing methods often directly set the pruned filters to zero during retraining,significantly reducing the parameter space.However,this direct pruning strategy frequently results in irreversible information loss.In the early stages of training,a network still contains much uncertainty,and evaluating filter importance may not be sufficiently rigorous.To manage the pruning process effectively,this paper proposes a flexible neural network pruning algorithm based on the logistic growth differential equation,considering the characteristics of network training.Unlike other pruning algorithms that directly reduce filter weights,this algorithm introduces a three-stage adaptive weight decay strategy inspired by the logistic growth differential equation.It employs a gentle decay rate in the initial training stage,a rapid decay rate during the intermediate stage,and a slower decay rate in the network convergence stage.Additionally,the decay rate is adjusted adaptively based on the filter weights at each stage.By controlling the adaptive decay rate at each stage,the pruning of neural network filters can be effectively managed.In experiments conducted on the CIFAR-10 and ILSVRC-2012 datasets,the pruning of neural networks significantly reduces the floating-point operations while maintaining the same pruning rate.Specifically,when implementing a 30%pruning rate on the ResNet-110 network,the pruned neural network not only decreases floating-point operations by 40.8%but also enhances the classification accuracy by 0.49%compared to the original network.展开更多
The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classificati...The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.展开更多
Filter pruning effectively compresses the neural network by reducing both its parameters and computational cost.Existing pruning methods typically rely on pre-designed pruning criteria to measure filter importance and...Filter pruning effectively compresses the neural network by reducing both its parameters and computational cost.Existing pruning methods typically rely on pre-designed pruning criteria to measure filter importance and remove those deemed unimportant.However,different layers of the neural network exhibit varying filter distributions,making it inappropriate to implement the same pruning criterion for all layers.Additionally,some approaches apply different criteria from the set of pre-defined pruning rules for different layers,but the limited space leads to the difficulty of covering all layers.If criteria for all layers are manually designed,it is costly and difficult to generalize to other networks.To solve this problem,we present a novel neural network pruning method based on the Criterion Learner and Attention Distillation(CLAD).Specifically,CLAD develops a differentiable criterion learner,which is integrated into each layer of the network.The learner can automatically learn the appropriate pruning criterion according to the filter parameters of each layer,thus the requirement of manual design is eliminated.Furthermore,the criterion learner is trained end-to-end by the gradient optimization algorithm to achieve efficient pruning.In addition,attention distillation,which fully utilizes the knowledge of unpruned networks to guide the optimization of the learner and improve the pruned network performance,is introduced in the process of learner optimization.Experiments conducted on various datasets and networks demonstrate the effectiveness of the proposed method.Notably,CLAD reduces the FLOPs of Res Net-110 by about 53%on the CIFAR-10 dataset,while simultaneously improves the network's accuracy by 0.05%.Moreover,it reduces the FLOPs of Res Net-50 by about 46%on the Image Net-1K dataset,and maintains a top-1 accuracy of 75.45%.展开更多
Accurate traffic flow prediction(TFP)is vital for efficient and sustainable transportation management and the development of intelligent traffic systems.However,missing data in real-world traffic datasets poses a sign...Accurate traffic flow prediction(TFP)is vital for efficient and sustainable transportation management and the development of intelligent traffic systems.However,missing data in real-world traffic datasets poses a significant challenge to maintaining prediction precision.This study introduces REPTF-TMDI,a novel method that combines a Reduced Error Pruning Tree Forest(REPTree Forest)with a newly proposed Time-based Missing Data Imputation(TMDI)approach.The REP Tree Forest,an ensemble learning approach,is tailored for time-related traffic data to enhance predictive accuracy and support the evolution of sustainable urbanmobility solutions.Meanwhile,the TMDI approach exploits temporal patterns to estimate missing values reliably whenever empty fields are encountered.The proposed method was evaluated using hourly traffic flow data from a major U.S.roadway spanning 2012-2018,incorporating temporal features(e.g.,hour,day,month,year,weekday),holiday indicator,and weather conditions(temperature,rain,snow,and cloud coverage).Experimental results demonstrated that the REPTF-TMDI method outperformed conventional imputation techniques across various missing data ratios by achieving an average 11.76%improvement in terms of correlation coefficient(R).Furthermore,REPTree Forest achieved improvements of 68.62%in RMSE and 70.52%in MAE compared to existing state-of-the-art models.These findings highlight the method’s ability to significantly boost traffic flow prediction accuracy,even in the presence of missing data,thereby contributing to the broader objectives of sustainable urban transportation systems.展开更多
The dynamic routing mechanism in evolvable networks enables adaptive reconfiguration of topol-ogical structures and transmission pathways based on real-time task requirements and data character-istics.However,the heig...The dynamic routing mechanism in evolvable networks enables adaptive reconfiguration of topol-ogical structures and transmission pathways based on real-time task requirements and data character-istics.However,the heightened architectural complexity and expanded parameter dimensionality in evolvable networks present significant implementation challenges when deployed in resource-con-strained environments.Due to the critical paths ignored,traditional pruning strategies cannot get a desired trade-off between accuracy and efficiency.For this reason,a critical path retention pruning(CPRP)method is proposed.By deeply traversing the computational graph,the dependency rela-tionship among nodes is derived.Then the nodes are grouped and sorted according to their contribu-tion value.The redundant operations are removed as much as possible while ensuring that the criti-cal path is not affected.As a result,computational efficiency is improved while a higher accuracy is maintained.On the CIFAR benchmark,the experimental results demonstrate that CPRP-induced pruning incurs accuracy degradation below 4.00%,while outperforming traditional feature-agnostic grouping methods by an average 8.98%accuracy improvement.Simultaneously,the pruned model attains a 2.41 times inference acceleration while achieving 48.92%parameter compression and 53.40%floating-point operations(FLOPs)reduction.展开更多
End-to-end object detection Transformer(DETR)successfully established the paradigm of the Transformer architecture in the field of object detection.Its end-to-end detection process and the idea of set prediction have ...End-to-end object detection Transformer(DETR)successfully established the paradigm of the Transformer architecture in the field of object detection.Its end-to-end detection process and the idea of set prediction have become one of the hottest network architectures in recent years.There has been an abundance of work improving upon DETR.However,DETR and its variants require a substantial amount of memory resources and computational costs,and the vast number of parameters in these networks is unfavorable for model deployment.To address this issue,a greedy pruning(GP)algorithm is proposed,applied to a variant denoising-DETR(DN-DETR),which can eliminate redundant parameters in the Transformer architecture of DN-DETR.Considering the different roles of the multi-head attention(MHA)module and the feed-forward network(FFN)module in the Transformer architecture,a modular greedy pruning(MGP)algorithm is proposed.This algorithm separates the two modules and applies their respective optimal strategies and parameters.The effectiveness of the proposed algorithm is validated on the COCO 2017 dataset.The model obtained through the MGP algorithm reduces the parameters by 49%and the number of floating point operations(FLOPs)by 44%compared to the Transformer architecture of DN-DETR.At the same time,the mean average precision(mAP)of the model increases from 44.1%to 45.3%.展开更多
3D sparse convolution has emerged as a pivotal technique for efficient voxel-based perception in autonomous systems,enabling selective feature extraction from non-empty voxels while suppressing computational waste.Des...3D sparse convolution has emerged as a pivotal technique for efficient voxel-based perception in autonomous systems,enabling selective feature extraction from non-empty voxels while suppressing computational waste.Despite its theoretical efficiency advantages,practical implementations face under-explored limitations:the fixed geometric patterns of conventional sparse convolutional kernels inevitably process non-contributory positions during sliding-window operations,particularly in regions with uneven point cloud density.To address this,we propose Hierarchical Shape Pruning for 3D Sparse Convolution(HSP-S),which dynamically eliminates redundant kernel stripes through layer-adaptive thresholding.Unlike static soft pruning methods,HSP-S maintains trainable sparsity patterns by progressively adjusting pruning thresholds during optimization,enlarging original parameter search space while removing redundant operations.Extensive experiments validate effectiveness of HSP-S acrossmajor autonomous driving benchmarks.On KITTI’s 3D object detection task,our method reduces 93.47%redundant kernel computations whilemaintaining comparable accuracy(1.56%mAP drop).Remarkably,on themore complexNuScenes benchmark,HSP-S achieves simultaneous computation reduction(21.94%sparsity)and accuracy gains(1.02%mAP(mean Average Precision)and 0.47%NDS(nuScenes detection score)improvement),demonstrating its scalability to diverse perception scenarios.This work establishes the first learnable shape pruning framework that simultaneously enhances computational efficiency and preserves detection accuracy in 3D perception systems.展开更多
[Objectives]This study was conducted to screen out reasonable pruning methods of walnut,and provide practical guidance for high-yield cultivation of walnut.[Methods]Xinfeng and Xinguang were used to study the effects ...[Objectives]This study was conducted to screen out reasonable pruning methods of walnut,and provide practical guidance for high-yield cultivation of walnut.[Methods]Xinfeng and Xinguang were used to study the effects of mechanical and artificial pruning methods on shoot growth,chlorophyll content in leaves,net photosynthetic rate and fruit quality.[Results]The results showed that:①the pruning method had a significant impact on the number of new shoots,and the number of new shoots of mechanical pruning was significantly higher than that of manual pruning;②the pruning method had a significant impact on the chlorophyll content,and the chlorophyll content of Xinguang of mechanical pruning was significantly higher than that of manual pruning;③the pruning method had a significant impact on the net photosynthetic rate of leaves,and the net photosynthetic rate of manual pruning was significantly higher than that of manual pruning;④Pruning methods had a significant effect on the number of walnut fruit,and the fruit trees pruned manually were significantly higher than those pruned mechanically;⑤the pruning method had no significant impact on the single fruit weight;⑥Pruning methods had a significant effect on the shell yield of a single walnut plant.The shell yield of single walnut plant pruned manually was significantly higher than that pruned mechanically;⑦pruning mode had a significant impact on the yield of walnut per plant,and the yield of artificially pruned walnut per plant was significantly higher than that of mechanical pruning;and⑧Xinfeng s chlorophyll content,net photosynthetic rate,fruit number,shell yield per plant and kernel yield per plant were all better than that of Xinguang,in addition,the growth and development of new shoots,the fruit quantity and quality of fruit also were affected by the interaction effect of genotype and pruning mode×variety.[Conclusions]It can be seen that different pruning methods have significant effects on the growth and development of new shoots and fruit yield and quality of walnut.Artificial pruning is more suitable for walnut cultivation by adjusting photosynthesis and improving the yield and quality of walnut;and Xinfeng is more suitable for popularization and production.展开更多
In the Yangtze River region, thepepper long-season cultivation needs oversummer pruning. Inorder to study the effects of different pruning treatments on growth of over-summer grown pepper, three differentpruning treat...In the Yangtze River region, thepepper long-season cultivation needs oversummer pruning. Inorder to study the effects of different pruning treatments on growth of over-summer grown pepper, three differentpruning treatments were applied to pepper seedlings, such as treatment one（re-growth pruning）, treatment two（four stems） and treatment three（multi-stem pruning） with no pruning as control（CK）.The results showed that themulti-stem pruning treatment enabling the plant to maintain the good plant type for high yield and reduce thevirus disease to improve the marketability of pepper fruits, compared with the other two pruning treatments;andthe yield and economic benefit were increased by 15.8% and 19%, compared with the control, respectively. Ingeneral the multi-stem pruning was the best for over-summer pepper cultivation.展开更多
Jujube(Ziziphus jujube Mill.)is a traditional economic forest crop and is widely cultivated in hilly areas of the Loess Plateau,China.However,soil desiccation was discovered in jujube plantations.Pruning is recognized...Jujube(Ziziphus jujube Mill.)is a traditional economic forest crop and is widely cultivated in hilly areas of the Loess Plateau,China.However,soil desiccation was discovered in jujube plantations.Pruning is recognized as a water-saving method that can reduces soil water consumption.In this study,we monitored the jujube plots with control(CK),light(C1),medium(C2)and high(C3)pruning intensities during the jujube growing period of 2012-2015 to explore the effect of pruning intensity on soil moisture and water use efficiency(WUE)of jujube plantations in the hilly Loess Plateau Region.The results showed that pruning is an effective method for soil water conservation in jujube plantations.Soil moisture increased with increasing pruning intensity during the jujube growing period of 2012-2015.C1,C2 and C3 pruning intensities increased soil water storage by 6.1-18.3,14.4-40.0 and 24.3-63.3 mm,respectively,compared to CK pruning intensity.Pruning promoted soil moisture infiltration to deeper soil layer.Soil moisture infiltrated to soil depths of 240,280 and>300 cm under C3 pruning intensity,220,260 and 260 cm under C2 pruning intensity,200,240 and 220 cm under C1 pruning intensity,and 180,200 and 160 cm under CK pruning intensity in 2013,2014 and 2015,respectively.Soil water deficit was alleviated by higher pruning intensity.In 2013-2015,soil water change was positive under C2(6.4 mm)and C3(26.8 mm)pruning intensities but negative under C1(-20.5 mm)and CK(-40.6 mm)pruning intensities.Moreover,pruning significantly improved fresh fruit yield and WUE of jujube plants.Fresh fruit yields were highest under C1 pruning intensity with the values of 6897.1-13,059.3 kg/hm^2,which were 2758.4-4712.8,385.7-1432.1 and 802.8-2331.5 kg/hm2 higher than those under CK,C2,and C3 pruning intensities during the jujube growing period of 2012-2015,respectively.However,C3 pruning intensity had the highest WUE values of 2.92-3.13 kg/m3,which were 1.6-2.0,1.1-1.2 and 1.0-1.1 times greater than those under CK,C1 and C2 pruning intensities,respectively.Therefore,C3 pruning intensity is recommended to jujube plantations for its economic and ecological benefits.These results provide an alternative strategy to mitigate soil desiccation in jujube plantations in the hilly Loess Plateau Region,which is critical for sustainable cultivation of economic forest trees in this region.展开更多
With the increment of the number of Gaussian components, the computation cost increases in the Gaussian mixture probability hypothesis density(GM-PHD) filter. Based on the theory of Chen et al, we propose an improved ...With the increment of the number of Gaussian components, the computation cost increases in the Gaussian mixture probability hypothesis density(GM-PHD) filter. Based on the theory of Chen et al, we propose an improved pruning algorithm for the GM-PHD filter, which utilizes not only the Gaussian components’ means and covariance, but their weights as a new criterion to improve the estimate accuracy of the conventional pruning algorithm for tracking very closely proximity targets. Moreover, it solves the end-less while-loop problem without the need of a second merging step. Simulation results show that this improved algorithm is easier to implement and more robust than the formal ones.展开更多
Two pot experiments were conducted to study the effects of root pruning at the stem elongation stage on non-hydraulic root-sourced signals （nHRS）, drought tolerance and water use efficiency of winter wheat （Triticu...Two pot experiments were conducted to study the effects of root pruning at the stem elongation stage on non-hydraulic root-sourced signals （nHRS）, drought tolerance and water use efficiency of winter wheat （Triticum aestivum）. The root pruning significantly reduced the root weight of wheat, but had no effect on root/shoot ratio at the two tested stages. At booting stage, specific root respiration of root pruned plants was significantly higher than those with intact roots （1.06 and 0.94 mmol g-1 s-1, respectively）. The soil water content （SWC） at which nHRS for root pruned plants appeared was higher and terminated lower than for intact root plants, the threshold range of nHRS was markedly greater for root pruned plants （61.1-44.6% field water capacity） than for intact root plants （57.9-46.1% field water capacity）. At flowering stage, while there was no significant difference in specific root respiration. The SWCs at which nHRS appeared and terminated were both higher for root pruned plants than for intact root plants. The values of chlorophyll fluorescence parameters, i.e., the effective photosystem II quantum yield （F PS II ）, the maximum photochemical efficiency of PS II （F v /F m ）, coefficient of photochemical quenching （qP）, and coefficient of non-photochemical quenching （NPQ）, in root pruned plants were significantly higher than in intact root plants, 7 d after withholding of water. Root pruned plants had significantly higher water use efficiency （WUE） than intact root plants in well-watered and medium drought soil, but not in severe drought condition. In addition, root pruning had no significant effect on grain yield in well-watered and medium drought soil, but significantly decreased grain yield in severe drought condition. In conclusion, the current study showed that root pruning significantly altered nHRS sensitivity and improved WUE of winter wheat in well-watered and medium drought soil, but lowered drought tolerance of winter wheat in severe drought soil. This suggests a possible direction of drought- resistance breeding and potential agricultural measure to improve WUE of winter wheat under semiarid conditions.展开更多
The effect of first thinning and pruning on height, diameter at breast height （DBH）, and volume growth was studied in individual trees of Pinus patula Schiede and Deppe in Chongoni Plantation, using four plots for t...The effect of first thinning and pruning on height, diameter at breast height （DBH）, and volume growth was studied in individual trees of Pinus patula Schiede and Deppe in Chongoni Plantation, using four plots for thinning trials. Each of the plots was 0.5 ha and subjected to one of the fol- lowing silvicultural treatments： first thinning and pruning, first thinning and no pruning, pruning and no thinning, and control （no pruning and no thinning）. The silvicultural treatments were randomized in four replicates. Fourteen years after planting, the following parameters were measured including total height, DBH, and volume. The highest DBH and volume growth was observed in thinning and pruning, while the highest growth in height was observed where there was pruning and no thinning. Clearly, both thinning and pruning are an important manage- ment option in pine species plantations in Malawi to maximize the increase in volume productivity.展开更多
A litterbag experiment of 12 weeks was conducted to study nitrogenmineralization process of prunings of six nitrogen-fixing hedgerowspecies in a dry valley of the Jinsha River. Prunigns wereincorporated into soil or u...A litterbag experiment of 12 weeks was conducted to study nitrogenmineralization process of prunings of six nitrogen-fixing hedgerowspecies in a dry valley of the Jinsha River. Prunigns wereincorporated into soil or used as mulch. The results indicated thatpruning N of the six hedgerow species was mineralized fast in thefirst week and then decreased slowly in the rest of the study period.When prunings were incorporated into soil, the amount of nitrogenmineralized by the end of the first week accounted for 69.9/100,58.2/100, 54.5/100, 43.0/100, 29.6/100 and 20.6/100 of the total N inprungins of Desmodium rensonii, Tephrosia candida, Leucaenaleucoephala, Albizia yunnanensis, Acacia dealbata, and Acaciamearnsii, respectively.展开更多
Microglia are the main non-neuronal cells in the central nervous system that have important roles in brain development and functional connectivity of neural circuits.In brain physiology,highly dynamic microglial proce...Microglia are the main non-neuronal cells in the central nervous system that have important roles in brain development and functional connectivity of neural circuits.In brain physiology,highly dynamic microglial processes are facilitated to sense the surrounding environment and stimuli.Once the brain switches its functional states,microglia are recruited to specific sites to exert their immune functions,including the release of cytokines and phagocytosis of cellular debris.The crosstalk of microglia between neurons,neural stem cells,endothelial cells,oligodendrocytes,and astrocytes contributes to their functions in synapse pruning,neurogenesis,vascularization,myelination,and blood-brain barrier permeability.In this review,we highlight the neuron-derived“find-me,”“eat-me,”and“don't eat-me”molecular signals that drive microglia in response to changes in neuronal activity for synapse refinement during brain development.This review reveals the molecular mechanism of neuron-microglia interaction in synaptic pruning and presents novel ideas for the synaptic pruning of microglia in disease,thereby providing important clues for discovery of target drugs and development of nervous system disease treatment methods targeting synaptic dysfunction.展开更多
As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorit...As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.展开更多
基金supported by the National Natural Science Foundation of China,No.31760290,82160688the Key Development Areas Project of Ganzhou Science and Technology,No.2022B-SF9554(all to XL)。
文摘Synaptic pruning is a crucial process in synaptic refinement,eliminating unstable synaptic connections in neural circuits.This process is triggered and regulated primarily by spontaneous neural activity and experience-dependent mechanisms.The pruning process involves multiple molecular signals and a series of regulatory activities governing the“eat me”and“don't eat me”states.Under physiological conditions,the interaction between glial cells and neurons results in the clearance of unnecessary synapses,maintaining normal neural circuit functionality via synaptic pruning.Alterations in genetic and environmental factors can lead to imbalanced synaptic pruning,thus promoting the occurrence and development of autism spectrum disorder,schizophrenia,Alzheimer's disease,and other neurological disorders.In this review,we investigated the molecular mechanisms responsible for synaptic pruning during neural development.We focus on how synaptic pruning can regulate neural circuits and its association with neurological disorders.Furthermore,we discuss the application of emerging optical and imaging technologies to observe synaptic structure and function,as well as their potential for clinical translation.Our aim was to enhance our understanding of synaptic pruning during neural development,including the molecular basis underlying the regulation of synaptic function and the dynamic changes in synaptic density,and to investigate the potential role of these mechanisms in the pathophysiology of neurological diseases,thus providing a theoretical foundation for the treatment of neurological disorders.
基金supported by the National Natural Science Foundation of China under Grant No.62172132.
文摘The surge of large-scale models in recent years has led to breakthroughs in numerous fields,but it has also introduced higher computational costs and more complex network architectures.These increasingly large and intricate networks pose challenges for deployment and execution while also exacerbating the issue of network over-parameterization.To address this issue,various network compression techniques have been developed,such as network pruning.A typical pruning algorithm follows a three-step pipeline involving training,pruning,and retraining.Existing methods often directly set the pruned filters to zero during retraining,significantly reducing the parameter space.However,this direct pruning strategy frequently results in irreversible information loss.In the early stages of training,a network still contains much uncertainty,and evaluating filter importance may not be sufficiently rigorous.To manage the pruning process effectively,this paper proposes a flexible neural network pruning algorithm based on the logistic growth differential equation,considering the characteristics of network training.Unlike other pruning algorithms that directly reduce filter weights,this algorithm introduces a three-stage adaptive weight decay strategy inspired by the logistic growth differential equation.It employs a gentle decay rate in the initial training stage,a rapid decay rate during the intermediate stage,and a slower decay rate in the network convergence stage.Additionally,the decay rate is adjusted adaptively based on the filter weights at each stage.By controlling the adaptive decay rate at each stage,the pruning of neural network filters can be effectively managed.In experiments conducted on the CIFAR-10 and ILSVRC-2012 datasets,the pruning of neural networks significantly reduces the floating-point operations while maintaining the same pruning rate.Specifically,when implementing a 30%pruning rate on the ResNet-110 network,the pruned neural network not only decreases floating-point operations by 40.8%but also enhances the classification accuracy by 0.49%compared to the original network.
文摘The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
基金supported in part by the National Natural Science Foundation of China under grants 62073085,61973330 and 62350055in part by the Shenzhen Science and Technology Program,China under grant JCYJ20230807093513027in part by the Fundamental Research Funds for the Central Universities,China under grant 1243300008。
文摘Filter pruning effectively compresses the neural network by reducing both its parameters and computational cost.Existing pruning methods typically rely on pre-designed pruning criteria to measure filter importance and remove those deemed unimportant.However,different layers of the neural network exhibit varying filter distributions,making it inappropriate to implement the same pruning criterion for all layers.Additionally,some approaches apply different criteria from the set of pre-defined pruning rules for different layers,but the limited space leads to the difficulty of covering all layers.If criteria for all layers are manually designed,it is costly and difficult to generalize to other networks.To solve this problem,we present a novel neural network pruning method based on the Criterion Learner and Attention Distillation(CLAD).Specifically,CLAD develops a differentiable criterion learner,which is integrated into each layer of the network.The learner can automatically learn the appropriate pruning criterion according to the filter parameters of each layer,thus the requirement of manual design is eliminated.Furthermore,the criterion learner is trained end-to-end by the gradient optimization algorithm to achieve efficient pruning.In addition,attention distillation,which fully utilizes the knowledge of unpruned networks to guide the optimization of the learner and improve the pruned network performance,is introduced in the process of learner optimization.Experiments conducted on various datasets and networks demonstrate the effectiveness of the proposed method.Notably,CLAD reduces the FLOPs of Res Net-110 by about 53%on the CIFAR-10 dataset,while simultaneously improves the network's accuracy by 0.05%.Moreover,it reduces the FLOPs of Res Net-50 by about 46%on the Image Net-1K dataset,and maintains a top-1 accuracy of 75.45%.
文摘Accurate traffic flow prediction(TFP)is vital for efficient and sustainable transportation management and the development of intelligent traffic systems.However,missing data in real-world traffic datasets poses a significant challenge to maintaining prediction precision.This study introduces REPTF-TMDI,a novel method that combines a Reduced Error Pruning Tree Forest(REPTree Forest)with a newly proposed Time-based Missing Data Imputation(TMDI)approach.The REP Tree Forest,an ensemble learning approach,is tailored for time-related traffic data to enhance predictive accuracy and support the evolution of sustainable urbanmobility solutions.Meanwhile,the TMDI approach exploits temporal patterns to estimate missing values reliably whenever empty fields are encountered.The proposed method was evaluated using hourly traffic flow data from a major U.S.roadway spanning 2012-2018,incorporating temporal features(e.g.,hour,day,month,year,weekday),holiday indicator,and weather conditions(temperature,rain,snow,and cloud coverage).Experimental results demonstrated that the REPTF-TMDI method outperformed conventional imputation techniques across various missing data ratios by achieving an average 11.76%improvement in terms of correlation coefficient(R).Furthermore,REPTree Forest achieved improvements of 68.62%in RMSE and 70.52%in MAE compared to existing state-of-the-art models.These findings highlight the method’s ability to significantly boost traffic flow prediction accuracy,even in the presence of missing data,thereby contributing to the broader objectives of sustainable urban transportation systems.
基金Supported by the National Key Research and Development Program of China(No.2022ZD0119003)and the National Natural Science Founda-tion of China(No.61834005).
文摘The dynamic routing mechanism in evolvable networks enables adaptive reconfiguration of topol-ogical structures and transmission pathways based on real-time task requirements and data character-istics.However,the heightened architectural complexity and expanded parameter dimensionality in evolvable networks present significant implementation challenges when deployed in resource-con-strained environments.Due to the critical paths ignored,traditional pruning strategies cannot get a desired trade-off between accuracy and efficiency.For this reason,a critical path retention pruning(CPRP)method is proposed.By deeply traversing the computational graph,the dependency rela-tionship among nodes is derived.Then the nodes are grouped and sorted according to their contribu-tion value.The redundant operations are removed as much as possible while ensuring that the criti-cal path is not affected.As a result,computational efficiency is improved while a higher accuracy is maintained.On the CIFAR benchmark,the experimental results demonstrate that CPRP-induced pruning incurs accuracy degradation below 4.00%,while outperforming traditional feature-agnostic grouping methods by an average 8.98%accuracy improvement.Simultaneously,the pruned model attains a 2.41 times inference acceleration while achieving 48.92%parameter compression and 53.40%floating-point operations(FLOPs)reduction.
基金Shanghai Municipal Commission of Economy and Information Technology,China(No.202301054)。
文摘End-to-end object detection Transformer(DETR)successfully established the paradigm of the Transformer architecture in the field of object detection.Its end-to-end detection process and the idea of set prediction have become one of the hottest network architectures in recent years.There has been an abundance of work improving upon DETR.However,DETR and its variants require a substantial amount of memory resources and computational costs,and the vast number of parameters in these networks is unfavorable for model deployment.To address this issue,a greedy pruning(GP)algorithm is proposed,applied to a variant denoising-DETR(DN-DETR),which can eliminate redundant parameters in the Transformer architecture of DN-DETR.Considering the different roles of the multi-head attention(MHA)module and the feed-forward network(FFN)module in the Transformer architecture,a modular greedy pruning(MGP)algorithm is proposed.This algorithm separates the two modules and applies their respective optimal strategies and parameters.The effectiveness of the proposed algorithm is validated on the COCO 2017 dataset.The model obtained through the MGP algorithm reduces the parameters by 49%and the number of floating point operations(FLOPs)by 44%compared to the Transformer architecture of DN-DETR.At the same time,the mean average precision(mAP)of the model increases from 44.1%to 45.3%.
文摘3D sparse convolution has emerged as a pivotal technique for efficient voxel-based perception in autonomous systems,enabling selective feature extraction from non-empty voxels while suppressing computational waste.Despite its theoretical efficiency advantages,practical implementations face under-explored limitations:the fixed geometric patterns of conventional sparse convolutional kernels inevitably process non-contributory positions during sliding-window operations,particularly in regions with uneven point cloud density.To address this,we propose Hierarchical Shape Pruning for 3D Sparse Convolution(HSP-S),which dynamically eliminates redundant kernel stripes through layer-adaptive thresholding.Unlike static soft pruning methods,HSP-S maintains trainable sparsity patterns by progressively adjusting pruning thresholds during optimization,enlarging original parameter search space while removing redundant operations.Extensive experiments validate effectiveness of HSP-S acrossmajor autonomous driving benchmarks.On KITTI’s 3D object detection task,our method reduces 93.47%redundant kernel computations whilemaintaining comparable accuracy(1.56%mAP drop).Remarkably,on themore complexNuScenes benchmark,HSP-S achieves simultaneous computation reduction(21.94%sparsity)and accuracy gains(1.02%mAP(mean Average Precision)and 0.47%NDS(nuScenes detection score)improvement),demonstrating its scalability to diverse perception scenarios.This work establishes the first learnable shape pruning framework that simultaneously enhances computational efficiency and preserves detection accuracy in 3D perception systems.
基金the Project of Demonstration of Walnut Fine Variety Promotion and Standardized Management Technology in the Technology Promotion Demonstration Project of Central Finance Forest and Grass Science in 2020[Xin[2020]No.TG12].
文摘[Objectives]This study was conducted to screen out reasonable pruning methods of walnut,and provide practical guidance for high-yield cultivation of walnut.[Methods]Xinfeng and Xinguang were used to study the effects of mechanical and artificial pruning methods on shoot growth,chlorophyll content in leaves,net photosynthetic rate and fruit quality.[Results]The results showed that:①the pruning method had a significant impact on the number of new shoots,and the number of new shoots of mechanical pruning was significantly higher than that of manual pruning;②the pruning method had a significant impact on the chlorophyll content,and the chlorophyll content of Xinguang of mechanical pruning was significantly higher than that of manual pruning;③the pruning method had a significant impact on the net photosynthetic rate of leaves,and the net photosynthetic rate of manual pruning was significantly higher than that of manual pruning;④Pruning methods had a significant effect on the number of walnut fruit,and the fruit trees pruned manually were significantly higher than those pruned mechanically;⑤the pruning method had no significant impact on the single fruit weight;⑥Pruning methods had a significant effect on the shell yield of a single walnut plant.The shell yield of single walnut plant pruned manually was significantly higher than that pruned mechanically;⑦pruning mode had a significant impact on the yield of walnut per plant,and the yield of artificially pruned walnut per plant was significantly higher than that of mechanical pruning;and⑧Xinfeng s chlorophyll content,net photosynthetic rate,fruit number,shell yield per plant and kernel yield per plant were all better than that of Xinguang,in addition,the growth and development of new shoots,the fruit quantity and quality of fruit also were affected by the interaction effect of genotype and pruning mode×variety.[Conclusions]It can be seen that different pruning methods have significant effects on the growth and development of new shoots and fruit yield and quality of walnut.Artificial pruning is more suitable for walnut cultivation by adjusting photosynthesis and improving the yield and quality of walnut;and Xinfeng is more suitable for popularization and production.
基金Supported by National Key Technology R&D Program(2014BAD05B04)The Expert Program of Cultivation Post for Hunan Vegetable Industry Technology System基金项目国家科技支撑课题(2014BAD-05B04)~~
文摘In the Yangtze River region, thepepper long-season cultivation needs oversummer pruning. Inorder to study the effects of different pruning treatments on growth of over-summer grown pepper, three differentpruning treatments were applied to pepper seedlings, such as treatment one（re-growth pruning）, treatment two（four stems） and treatment three（multi-stem pruning） with no pruning as control（CK）.The results showed that themulti-stem pruning treatment enabling the plant to maintain the good plant type for high yield and reduce thevirus disease to improve the marketability of pepper fruits, compared with the other two pruning treatments;andthe yield and economic benefit were increased by 15.8% and 19%, compared with the control, respectively. Ingeneral the multi-stem pruning was the best for over-summer pepper cultivation.
基金supported by the China National Support Program(2015BAC01B03)the Shaanxi Co-ordination Program(2016KTZDNY0105)the National Key Research and Development Program of China(2016YED0300201)
文摘Jujube(Ziziphus jujube Mill.)is a traditional economic forest crop and is widely cultivated in hilly areas of the Loess Plateau,China.However,soil desiccation was discovered in jujube plantations.Pruning is recognized as a water-saving method that can reduces soil water consumption.In this study,we monitored the jujube plots with control(CK),light(C1),medium(C2)and high(C3)pruning intensities during the jujube growing period of 2012-2015 to explore the effect of pruning intensity on soil moisture and water use efficiency(WUE)of jujube plantations in the hilly Loess Plateau Region.The results showed that pruning is an effective method for soil water conservation in jujube plantations.Soil moisture increased with increasing pruning intensity during the jujube growing period of 2012-2015.C1,C2 and C3 pruning intensities increased soil water storage by 6.1-18.3,14.4-40.0 and 24.3-63.3 mm,respectively,compared to CK pruning intensity.Pruning promoted soil moisture infiltration to deeper soil layer.Soil moisture infiltrated to soil depths of 240,280 and>300 cm under C3 pruning intensity,220,260 and 260 cm under C2 pruning intensity,200,240 and 220 cm under C1 pruning intensity,and 180,200 and 160 cm under CK pruning intensity in 2013,2014 and 2015,respectively.Soil water deficit was alleviated by higher pruning intensity.In 2013-2015,soil water change was positive under C2(6.4 mm)and C3(26.8 mm)pruning intensities but negative under C1(-20.5 mm)and CK(-40.6 mm)pruning intensities.Moreover,pruning significantly improved fresh fruit yield and WUE of jujube plants.Fresh fruit yields were highest under C1 pruning intensity with the values of 6897.1-13,059.3 kg/hm^2,which were 2758.4-4712.8,385.7-1432.1 and 802.8-2331.5 kg/hm2 higher than those under CK,C2,and C3 pruning intensities during the jujube growing period of 2012-2015,respectively.However,C3 pruning intensity had the highest WUE values of 2.92-3.13 kg/m3,which were 1.6-2.0,1.1-1.2 and 1.0-1.1 times greater than those under CK,C1 and C2 pruning intensities,respectively.Therefore,C3 pruning intensity is recommended to jujube plantations for its economic and ecological benefits.These results provide an alternative strategy to mitigate soil desiccation in jujube plantations in the hilly Loess Plateau Region,which is critical for sustainable cultivation of economic forest trees in this region.
基金supported by the National Natural Science Foundation of China(61703228)
文摘With the increment of the number of Gaussian components, the computation cost increases in the Gaussian mixture probability hypothesis density(GM-PHD) filter. Based on the theory of Chen et al, we propose an improved pruning algorithm for the GM-PHD filter, which utilizes not only the Gaussian components’ means and covariance, but their weights as a new criterion to improve the estimate accuracy of the conventional pruning algorithm for tracking very closely proximity targets. Moreover, it solves the end-less while-loop problem without the need of a second merging step. Simulation results show that this improved algorithm is easier to implement and more robust than the formal ones.
基金supported by the Fund of State Key Laboratory of Soil Erosion and Dryland Farming on the Loess Plateau,China(10501-1201)the Key Technologies R&D Program of China during the 11th Five-Year Plan period(2012BAD14B08)the Innovation Team Program,Ministry of Education of China
文摘Two pot experiments were conducted to study the effects of root pruning at the stem elongation stage on non-hydraulic root-sourced signals （nHRS）, drought tolerance and water use efficiency of winter wheat （Triticum aestivum）. The root pruning significantly reduced the root weight of wheat, but had no effect on root/shoot ratio at the two tested stages. At booting stage, specific root respiration of root pruned plants was significantly higher than those with intact roots （1.06 and 0.94 mmol g-1 s-1, respectively）. The soil water content （SWC） at which nHRS for root pruned plants appeared was higher and terminated lower than for intact root plants, the threshold range of nHRS was markedly greater for root pruned plants （61.1-44.6% field water capacity） than for intact root plants （57.9-46.1% field water capacity）. At flowering stage, while there was no significant difference in specific root respiration. The SWCs at which nHRS appeared and terminated were both higher for root pruned plants than for intact root plants. The values of chlorophyll fluorescence parameters, i.e., the effective photosystem II quantum yield （F PS II ）, the maximum photochemical efficiency of PS II （F v /F m ）, coefficient of photochemical quenching （qP）, and coefficient of non-photochemical quenching （NPQ）, in root pruned plants were significantly higher than in intact root plants, 7 d after withholding of water. Root pruned plants had significantly higher water use efficiency （WUE） than intact root plants in well-watered and medium drought soil, but not in severe drought condition. In addition, root pruning had no significant effect on grain yield in well-watered and medium drought soil, but significantly decreased grain yield in severe drought condition. In conclusion, the current study showed that root pruning significantly altered nHRS sensitivity and improved WUE of winter wheat in well-watered and medium drought soil, but lowered drought tolerance of winter wheat in severe drought soil. This suggests a possible direction of drought- resistance breeding and potential agricultural measure to improve WUE of winter wheat under semiarid conditions.
文摘The effect of first thinning and pruning on height, diameter at breast height （DBH）, and volume growth was studied in individual trees of Pinus patula Schiede and Deppe in Chongoni Plantation, using four plots for thinning trials. Each of the plots was 0.5 ha and subjected to one of the fol- lowing silvicultural treatments： first thinning and pruning, first thinning and no pruning, pruning and no thinning, and control （no pruning and no thinning）. The silvicultural treatments were randomized in four replicates. Fourteen years after planting, the following parameters were measured including total height, DBH, and volume. The highest DBH and volume growth was observed in thinning and pruning, while the highest growth in height was observed where there was pruning and no thinning. Clearly, both thinning and pruning are an important manage- ment option in pine species plantations in Malawi to maximize the increase in volume productivity.
基金Project supported jointly by the Chengdu Di Ao Science Foundation and the Sichuan Provincial Science Foundation for Young Scient
文摘A litterbag experiment of 12 weeks was conducted to study nitrogenmineralization process of prunings of six nitrogen-fixing hedgerowspecies in a dry valley of the Jinsha River. Prunigns wereincorporated into soil or used as mulch. The results indicated thatpruning N of the six hedgerow species was mineralized fast in thefirst week and then decreased slowly in the rest of the study period.When prunings were incorporated into soil, the amount of nitrogenmineralized by the end of the first week accounted for 69.9/100,58.2/100, 54.5/100, 43.0/100, 29.6/100 and 20.6/100 of the total N inprungins of Desmodium rensonii, Tephrosia candida, Leucaenaleucoephala, Albizia yunnanensis, Acacia dealbata, and Acaciamearnsii, respectively.
基金supported by the National Natural Science Foundation of ChinaNo.32200778(to QC)+5 种基金the Natural Science Foundation of Jiangsu ProvinceNo.BK20220494(to QC)Suzhou Medical and Health Technology Innovation ProjectNo.SKY2022107(to QC)a grant from the Clinical Research Center of Neurological Disease in The Second Affiliated Hospital of Soochow UniversityNos.ND2022A04(to QC)and ND2023B06(to JS)。
文摘Microglia are the main non-neuronal cells in the central nervous system that have important roles in brain development and functional connectivity of neural circuits.In brain physiology,highly dynamic microglial processes are facilitated to sense the surrounding environment and stimuli.Once the brain switches its functional states,microglia are recruited to specific sites to exert their immune functions,including the release of cytokines and phagocytosis of cellular debris.The crosstalk of microglia between neurons,neural stem cells,endothelial cells,oligodendrocytes,and astrocytes contributes to their functions in synapse pruning,neurogenesis,vascularization,myelination,and blood-brain barrier permeability.In this review,we highlight the neuron-derived“find-me,”“eat-me,”and“don't eat-me”molecular signals that drive microglia in response to changes in neuronal activity for synapse refinement during brain development.This review reveals the molecular mechanism of neuron-microglia interaction in synaptic pruning and presents novel ideas for the synaptic pruning of microglia in disease,thereby providing important clues for discovery of target drugs and development of nervous system disease treatment methods targeting synaptic dysfunction.
基金supported by the National Natural Science Foundation of China (61074127)
文摘As the solutions of the least squares support vector regression machine （LS-SVRM） are not sparse, it leads to slow prediction speed and limits its applications. The defects of the ex- isting adaptive pruning algorithm for LS-SVRM are that the training speed is slow, and the generalization performance is not satis- factory, especially for large scale problems. Hence an improved algorithm is proposed. In order to accelerate the training speed, the pruned data point and fast leave-one-out error are employed to validate the temporary model obtained after decremental learning. The novel objective function in the termination condition which in- volves the whole constraints generated by all training data points and three pruning strategies are employed to improve the generali- zation performance. The effectiveness of the proposed algorithm is tested on six benchmark datasets. The sparse LS-SVRM model has a faster training speed and better generalization performance.
