Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost ...Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.展开更多
Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNN...Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNNs).Since CNNs require considerable converters between analog crossbars and digital peripheral circuits,recent studies map the binary neural networks(BNNs)onto RRAM and binarize the weights to{+1,-1}.However,two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights.In this work,we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture.First,we spilt the weight matrix into several small matrices by clustering adjacent columns together.Second,we extract 1s'patterns,i.e.,the submatrices only containing 1s,from the small weight matrix,such that each final output can be represented by the sum of several patterns.Third,we map these patterns onto RRAM crossbars,including pattern computation crossbars(PCCs)and pattern accumulation crossbars(PACs).Finally,we compare the pattern representation with two mainstream representations and adopt the more area efficient one.The evaluation results demonstrate that our framework can save over 20%of crossbar area effectively,compared with two mainstream representations.展开更多
Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to deter...Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to determine the segregation characteristics of jetsam particles for both the homogeneous and heterogeneous binary mixtures in terms of segregation distance by correlating it to the various system parameters, viz. initial static bed height, height of a layer of particles above the bottom grid, superficial gas velocity and average particle size and/or particle densities of the mixture through the dimensional analysis. Correlation on the basis of Artificial Neural Network approach has also been developed with the above system parameters thereby authenticating the development of correlation by the former approach. The calculated values of the segregation distance obtained for both the homogeneous and heterogeneous binary mixtures from both the types of ftuidized beds （i.e. under the static bed condition and the ftuidized bed condition） have also been compared with each other.展开更多
文摘Binary neural networks(BNNs)show promising utilization in cost and power-restricted domains such as edge devices and mobile systems.This is due to its significantly less computation and storage demand,but at the cost of degraded performance.To close the accuracy gap,in this paper we propose to add a complementary activation function(AF)ahead of the sign based binarization,and rely on the genetic algorithm(GA)to automatically search for the ideal AFs.These AFs can help extract extra information from the input data in the forward pass,while allowing improved gradient approximation in the backward pass.Fifteen novel AFs are identified through our GA-based search,while most of them show improved performance(up to 2.54%on ImageNet)when testing on different datasets and network models.Interestingly,periodic functions are identified as a key component for most of the discovered AFs,which rarely exist in human designed AFs.Our method offers a novel approach for designing general and application-specific BNN architecture.GAAF will be released on GitHub.
基金partly supported by the National Key Research and Development Program of China under Grant No.2020AAA0130400Beijing Municipal Science and Technology Program of China under Grant No.Z201100004220007+2 种基金the National Natural Science Foundation of China under Grant No.62090021Beijing Academy of Artificial Intelligence(BAAI)Alibaba Innovative Research(AIR)Program.
文摘Resistive random access memory(RRAM)has been demonstrated to implement multiply-and-accumulate(MAC)operations using a highly parallel analog fashion,which dramatically accelerates the convolutional neural networks(CNNs).Since CNNs require considerable converters between analog crossbars and digital peripheral circuits,recent studies map the binary neural networks(BNNs)onto RRAM and binarize the weights to{+1,-1}.However,two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights.In this work,we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture.First,we spilt the weight matrix into several small matrices by clustering adjacent columns together.Second,we extract 1s'patterns,i.e.,the submatrices only containing 1s,from the small weight matrix,such that each final output can be represented by the sum of several patterns.Third,we map these patterns onto RRAM crossbars,including pattern computation crossbars(PCCs)and pattern accumulation crossbars(PACs).Finally,we compare the pattern representation with two mainstream representations and adopt the more area efficient one.The evaluation results demonstrate that our framework can save over 20%of crossbar area effectively,compared with two mainstream representations.
文摘Binary mixtures of irregular materials of different particle sizes and/or particle densities are fluidized in a 15-cm diameter column with a perforated plate distributor. An attempt has been made in this work to determine the segregation characteristics of jetsam particles for both the homogeneous and heterogeneous binary mixtures in terms of segregation distance by correlating it to the various system parameters, viz. initial static bed height, height of a layer of particles above the bottom grid, superficial gas velocity and average particle size and/or particle densities of the mixture through the dimensional analysis. Correlation on the basis of Artificial Neural Network approach has also been developed with the above system parameters thereby authenticating the development of correlation by the former approach. The calculated values of the segregation distance obtained for both the homogeneous and heterogeneous binary mixtures from both the types of ftuidized beds （i.e. under the static bed condition and the ftuidized bed condition） have also been compared with each other.
