Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost ...Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.展开更多
Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNN...Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNNs).Since CNNs require considerable converters between analog crossbars and digital peripheral circuits,recent studies map the binary neural networks(BNNs)onto RRAM and binarize the weights to{+1,-1}.However,two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights.In this work,we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture.First,we spilt the weight matrix into several small matrices by clustering adjacent columns together.Second,we extract 1s'patterns,i.e.,the submatrices only containing 1s,from the small weight matrix,such that each final output can be represented by the sum of several patterns.Third,we map these patterns onto RRAM crossbars,including pattern computation crossbars(PCCs)and pattern accumulation crossbars(PACs).Finally,we compare the pattern representation with two mainstream representations and adopt the more area efficient one.The evaluation results demonstrate that our framework can save over 20%of crossbar area effectively,compared with two mainstream representations.展开更多
Traditional classification methods for cervical cells heavily rely on manual feature extraction,constraining their versatility due to the intricacies of cytology images.Although deep learning approaches offer remarkab...Traditional classification methods for cervical cells heavily rely on manual feature extraction,constraining their versatility due to the intricacies of cytology images.Although deep learning approaches offer remarkable po-tential,they often sacrifice domain-specific knowledge,particularly the morphological patterns characterizing various cell subtypes during automated feature extraction.To bridge this gap,we introduce a novel hierarchical framework that integrates robust features from color,texture,and morphology with latent representations discovered by an improved attention-based multi-scale local binary convolutional neural networks(MS-LBCNN),designed to facilitate powerful feature extraction mechanism.We enhance the standard 6-class Bethesda system(TBS)classification by incorporating a coarse-to-refine fusion strategy,which optimizes the classification pro-cess.The proposed method is uniquely equipped to manage the complexities present in both individual and clustered cell images.Upon rigorous evaluation across three independent data cohorts,our method consistently surpassed existing state-of-the-art techniques.The experimental results indicated the potential of our method in enhancing the development of automation-aided diagnostic systems,and bolstering both the accuracy and ef-ficiency of cytology screening procedures.展开更多
Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to deter...Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to determine the segregation characteristics of jetsam particles for both the homogeneous and heterogeneous binary mixtures in terms of segregation distance by correlating it to the various system parameters, viz. initial static bed height, height of a layer of particles above the bottom grid, superficial gas velocity and average particle size and/or particle densities of the mixture through the dimensional analysis. Correlation on the basis of Artificial Neural Network approach has also been developed with the above system parameters thereby authenticating the development of correlation by the former approach. The calculated values of the segregation distance obtained for both the homogeneous and heterogeneous binary mixtures from both the types of ftuidized beds （i.e. under the static bed condition and the ftuidized bed condition） have also been compared with each other.展开更多
文摘Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.
基金partly supported by the National Key Research and Development Program of China under Grant No.2020AAA0130400Beijing Municipal Science and Technology Program of China under Grant No.Z201100004220007+2 种基金the National Natural Science Foundation of China under Grant No.62090021Beijing Academy of Artificial Intelligence(BAAI)Alibaba Innovative Research(AIR)Program.
文摘Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNNs).Since CNNs require considerable converters between analog crossbars and digital peripheral circuits,recent studies map the binary neural networks(BNNs)onto RRAM and binarize the weights to{+1,-1}.However,two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights.In this work,we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture.First,we spilt the weight matrix into several small matrices by clustering adjacent columns together.Second,we extract 1s'patterns,i.e.,the submatrices only containing 1s,from the small weight matrix,such that each final output can be represented by the sum of several patterns.Third,we map these patterns onto RRAM crossbars,including pattern computation crossbars(PCCs)and pattern accumulation crossbars(PACs).Finally,we compare the pattern representation with two mainstream representations and adopt the more area efficient one.The evaluation results demonstrate that our framework can save over 20%of crossbar area effectively,compared with two mainstream representations.
基金supported by the Beijing Capital Health Development Research Project[Grant no.2024-2-1031]the Beijing Municipal Natural Science Foundation[Grant no.7192105].
文摘Traditional classification methods for cervical cells heavily rely on manual feature extraction,constraining their versatility due to the intricacies of cytology images.Although deep learning approaches offer remarkable po-tential,they often sacrifice domain-specific knowledge,particularly the morphological patterns characterizing various cell subtypes during automated feature extraction.To bridge this gap,we introduce a novel hierarchical framework that integrates robust features from color,texture,and morphology with latent representations discovered by an improved attention-based multi-scale local binary convolutional neural networks(MS-LBCNN),designed to facilitate powerful feature extraction mechanism.We enhance the standard 6-class Bethesda system(TBS)classification by incorporating a coarse-to-refine fusion strategy,which optimizes the classification pro-cess.The proposed method is uniquely equipped to manage the complexities present in both individual and clustered cell images.Upon rigorous evaluation across three independent data cohorts,our method consistently surpassed existing state-of-the-art techniques.The experimental results indicated the potential of our method in enhancing the development of automation-aided diagnostic systems,and bolstering both the accuracy and ef-ficiency of cytology screening procedures.
文摘Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to determine the segregation characteristics of jetsam particles for both the homogeneous and heterogeneous binary mixtures in terms of segregation distance by correlating it to the various system parameters, viz. initial static bed height, height of a layer of particles above the bottom grid, superficial gas velocity and average particle size and/or particle densities of the mixture through the dimensional analysis. Correlation on the basis of Artificial Neural Network approach has also been developed with the above system parameters thereby authenticating the development of correlation by the former approach. The calculated values of the segregation distance obtained for both the homogeneous and heterogeneous binary mixtures from both the types of ftuidized beds （i.e. under the static bed condition and the ftuidized bed condition） have also been compared with each other.
