Approximations based on random Fourier features have recently emerged as an efficient and elegant method for designing large-scale machine learning tasks.Unlike approaches using the Nystr?m method,which randomly sampl...Approximations based on random Fourier features have recently emerged as an efficient and elegant method for designing large-scale machine learning tasks.Unlike approaches using the Nystr?m method,which randomly samples the training examples,we make use of random Fourier features,whose basis functions(i.e.,cosine and sine)are sampled from a distribution independent from the training sample set,to cluster preference data which appears extensively in recommender systems.Firstly,we propose a two-stage preference clustering framework.In this framework,we make use of random Fourier features to map the preference matrix into the feature matrix,soon afterwards,utilize the traditional k-means approach to cluster preference data in the transformed feature space.Compared with traditional preference clustering,our method solves the problem of insufficient memory and greatly improves the efficiency of the operation.Experiments on movie data sets containing 100000 ratings,show that the proposed method is more effective in clustering accuracy than the Nystr?m and k-means,while also achieving better performance than these clustering approaches.展开更多
As the core component of the transformer model,the attention has been proved as all you need in artificial intelligence field in recent years.However,conventional electronic processors are unable to cope with the expo...As the core component of the transformer model,the attention has been proved as all you need in artificial intelligence field in recent years.However,conventional electronic processors are unable to cope with the exponentially increasing hardware costs and energy consumption of the computing-expensive attention.While the photonic neural network(NN)chips provide alternative energy-efficient solutions for accelerating the matrix multiplication(MM),existing photonic accelerators are primarily designed for weight-static NNs that involve MM between the learned weight matrix and input tensors and thus are inefficient in supporting attention mechanisms that require dynamic input operands.Here we propose an attention mechanism relying solely on the runtime-programable optical-interference.Through theoretical analyses,numerical simulations and experimental validations,we demonstrate the photonic“all-interference”attention with learning capability equivalent to classical self-attention,and implement the photonic transformer chip(PTC).Evaluation shows that the PTC is promising to exceed 200 pera-operations per second(POPS)with 1POPS/mm2 computation density and 0.5 POPS/W power efficiency,much better than prior photonic accelerators,and delivers over 200×energy reduction and 2 to 3 orders of magnitude higher computation capability compared to the electronic counterpart.The photonic transformer with“all-interference”attention proposed in this work highlights the immense potential of photonics to construct its own computing paradigm for general purpose machine learning.展开更多
The simulation of lubrication on rough surfaces is essential for the design and optimization of tribological performance.Although the application of physics-informed neural networks(PINNs)in analyzing hydrodynamic lub...The simulation of lubrication on rough surfaces is essential for the design and optimization of tribological performance.Although the application of physics-informed neural networks(PINNs)in analyzing hydrodynamic lubrication has been increasing,their implementation has predominantly been restricted to smooth surfaces.This limitation arises from the inherent spectral bias of conventional PINN methodologies,which tend to prioritize the learning of low-frequency features,thereby hindering their ability to analyze rough surfaces characterized by high-frequency signals effectively.To date,there have been no reported instances of PINN methodologies being applied to rough surface lubrication.In response to these challenges,this paper presents an innovative multiscale lubrication neural network(MLNN)architecture that incorporates a trainable Fourier feature embedding.By integrating learnable feature embedding frequencies,this architecture is capable of automatically adjusting to various frequency components,thus improving the analysis of rough surface characteristics.The proposed method has been evaluated across a range of surface topographies,with results compared to those derived from the finite element method(FEM).The comparative analysis indicates a high degree of consistency between the MLNN results and those obtained through FEM.Moreover,this novel architecture demonstrates superior performance in terms of both accuracy and computational efficiency when compared to traditional Fourier feature networks that utilize fixed feature embedding frequencies.As a result,the MLNN model represents a more effective tool for the analysis of lubrication on rough surfaces.展开更多
Driven by the need of a plethora of machine learning applications,several attempts have been made at improving the performance of classifiers applied to imbalanced datasets.In this paper,we present a fast maximum entr...Driven by the need of a plethora of machine learning applications,several attempts have been made at improving the performance of classifiers applied to imbalanced datasets.In this paper,we present a fast maximum entropy machine(MEM)combined with a synthetic minority over-sampling technique for handling binary classification problems with high imbalance ratios,large numbers of data samples,and medium/large numbers of features.A random Fourier feature representation of kernel functions and primal estimated sub-gradient solver for support vector machine(PEGASOS)are applied to speed up the classic MEM.Experiments have been conducted using various real datasets(including two China Mobile datasets and several other standard test datasets)with various configurations.The obtained results demonstrate that the proposed algorithm has extremely low complexity but an excellent overall classification performance(in terms of several widely used evaluation metrics)as compared to the classic MEM and some other state-of-the-art methods.The proposed algorithm is particularly valuable in big data applications owing to its significantly low computational complexity.展开更多
基金supported by the National Natural Science Foundation of China(Nos.61872260 and 61592419)the Natural Science Foundation of Shanxi Province(No.201703D421013).
文摘Approximations based on random Fourier features have recently emerged as an efficient and elegant method for designing large-scale machine learning tasks.Unlike approaches using the Nystr?m method,which randomly samples the training examples,we make use of random Fourier features,whose basis functions(i.e.,cosine and sine)are sampled from a distribution independent from the training sample set,to cluster preference data which appears extensively in recommender systems.Firstly,we propose a two-stage preference clustering framework.In this framework,we make use of random Fourier features to map the preference matrix into the feature matrix,soon afterwards,utilize the traditional k-means approach to cluster preference data in the transformed feature space.Compared with traditional preference clustering,our method solves the problem of insufficient memory and greatly improves the efficiency of the operation.Experiments on movie data sets containing 100000 ratings,show that the proposed method is more effective in clustering accuracy than the Nystr?m and k-means,while also achieving better performance than these clustering approaches.
基金supported by the National Key Research and Development Program of China(2021YFB2801900,2021YFB2801901,2021YFB2801902,2021YFB2801904)National Natural Science Foundation of China(No.61974177)+1 种基金National Outstanding Youth Science Fund Project of National Natural Science Foundation of China(62022062)The Fundamental Research Funds for the Central Universities(QTZX23041).
文摘As the core component of the transformer model,the attention has been proved as all you need in artificial intelligence field in recent years.However,conventional electronic processors are unable to cope with the exponentially increasing hardware costs and energy consumption of the computing-expensive attention.While the photonic neural network(NN)chips provide alternative energy-efficient solutions for accelerating the matrix multiplication(MM),existing photonic accelerators are primarily designed for weight-static NNs that involve MM between the learned weight matrix and input tensors and thus are inefficient in supporting attention mechanisms that require dynamic input operands.Here we propose an attention mechanism relying solely on the runtime-programable optical-interference.Through theoretical analyses,numerical simulations and experimental validations,we demonstrate the photonic“all-interference”attention with learning capability equivalent to classical self-attention,and implement the photonic transformer chip(PTC).Evaluation shows that the PTC is promising to exceed 200 pera-operations per second(POPS)with 1POPS/mm2 computation density and 0.5 POPS/W power efficiency,much better than prior photonic accelerators,and delivers over 200×energy reduction and 2 to 3 orders of magnitude higher computation capability compared to the electronic counterpart.The photonic transformer with“all-interference”attention proposed in this work highlights the immense potential of photonics to construct its own computing paradigm for general purpose machine learning.
基金supported by the National Natural Science Foundation of China(Grant No.52130502)the National Key Laboratory of Marine Engine Science and Technology(Grant No.LAB2023-06-WD)。
文摘The simulation of lubrication on rough surfaces is essential for the design and optimization of tribological performance.Although the application of physics-informed neural networks(PINNs)in analyzing hydrodynamic lubrication has been increasing,their implementation has predominantly been restricted to smooth surfaces.This limitation arises from the inherent spectral bias of conventional PINN methodologies,which tend to prioritize the learning of low-frequency features,thereby hindering their ability to analyze rough surfaces characterized by high-frequency signals effectively.To date,there have been no reported instances of PINN methodologies being applied to rough surface lubrication.In response to these challenges,this paper presents an innovative multiscale lubrication neural network(MLNN)architecture that incorporates a trainable Fourier feature embedding.By integrating learnable feature embedding frequencies,this architecture is capable of automatically adjusting to various frequency components,thus improving the analysis of rough surface characteristics.The proposed method has been evaluated across a range of surface topographies,with results compared to those derived from the finite element method(FEM).The comparative analysis indicates a high degree of consistency between the MLNN results and those obtained through FEM.Moreover,this novel architecture demonstrates superior performance in terms of both accuracy and computational efficiency when compared to traditional Fourier feature networks that utilize fixed feature embedding frequencies.As a result,the MLNN model represents a more effective tool for the analysis of lubrication on rough surfaces.
基金The author Feng Yin was funded by the Shenzhen Science and Technology Innovation Council(No.JCYJ20170307155957688)and by National Natural Science Foundation of China Key Project(No.61731018)The authors Feng Yin and Shuguang(Robert)Cui were funded by Shenzhen Fundamental Research Funds under Grant(Key Lab)No.ZDSYS201707251409055,Grant(Peacock)No.KQTD2015033114415450,and Guangdong province“The Pearl River Talent Recruitment Program Innovative and Entrepreneurial Teams in 2017”-Data Driven Evolution of Future Intelligent Network Team.The associate editor coordinating the review of this paper and approving it for publication was X.Cheng.
文摘Driven by the need of a plethora of machine learning applications,several attempts have been made at improving the performance of classifiers applied to imbalanced datasets.In this paper,we present a fast maximum entropy machine(MEM)combined with a synthetic minority over-sampling technique for handling binary classification problems with high imbalance ratios,large numbers of data samples,and medium/large numbers of features.A random Fourier feature representation of kernel functions and primal estimated sub-gradient solver for support vector machine(PEGASOS)are applied to speed up the classic MEM.Experiments have been conducted using various real datasets(including two China Mobile datasets and several other standard test datasets)with various configurations.The obtained results demonstrate that the proposed algorithm has extremely low complexity but an excellent overall classification performance(in terms of several widely used evaluation metrics)as compared to the classic MEM and some other state-of-the-art methods.The proposed algorithm is particularly valuable in big data applications owing to its significantly low computational complexity.
