In this paper,the small-signal modeling of the Indium Phosphide High Electron Mobility Transistor(InP HEMT)based on the Transformer neural network model is investigated.The AC S-parameters of the HEMT device are train...In this paper,the small-signal modeling of the Indium Phosphide High Electron Mobility Transistor(InP HEMT)based on the Transformer neural network model is investigated.The AC S-parameters of the HEMT device are trained and validated using the Transformer model.In the proposed model,the eight-layer transformer encoders are connected in series and the encoder layer of each Transformer consists of the multi-head attention layer and the feed-forward neural network layer.The experimental results show that the measured and modeled S-parameters of the HEMT device match well in the frequency range of 0.5-40 GHz,with the errors versus frequency less than 1%.Compared with other models,good accuracy can be achieved to verify the effectiveness of the proposed model.展开更多
Cyberbullying on social media poses significant psychological risks,yet most detection systems over-simplify the task by focusing on binary classification,ignoring nuanced categories like passive-aggressive remarks or...Cyberbullying on social media poses significant psychological risks,yet most detection systems over-simplify the task by focusing on binary classification,ignoring nuanced categories like passive-aggressive remarks or indirect slurs.To address this gap,we propose a hybrid framework combining Term Frequency-Inverse Document Frequency(TF-IDF),word-to-vector(Word2Vec),and Bidirectional Encoder Representations from Transformers(BERT)based models for multi-class cyberbullying detection.Our approach integrates TF-IDF for lexical specificity and Word2Vec for semantic relationships,fused with BERT’s contextual embeddings to capture syntactic and semantic complexities.We evaluate the framework on a publicly available dataset of 47,000 annotated social media posts across five cyberbullying categories:age,ethnicity,gender,religion,and indirect aggression.Among BERT variants tested,BERT Base Un-Cased achieved the highest performance with 93%accuracy(standard deviation across±1%5-fold cross-validation)and an average AUC of 0.96,outperforming standalone TF-IDF(78%)and Word2Vec(82%)models.Notably,it achieved near-perfect AUC scores(0.99)for age and ethnicity-based bullying.A comparative analysis with state-of-the-art benchmarks,including Generative Pre-trained Transformer 2(GPT-2)and Text-to-Text Transfer Transformer(T5)models highlights BERT’s superiority in handling ambiguous language.This work advances cyberbullying detection by demonstrating how hybrid feature extraction and transformer models improve multi-class classification,offering a scalable solution for moderating nuanced harmful content.展开更多
AlphaPanda(AlphaFold2[1]inspired protein-specific antibody design in a diffusional manner)is an advanced algorithm for designing complementary determining regions(CDRs)of the antibody targeted the specific epitope,com...AlphaPanda(AlphaFold2[1]inspired protein-specific antibody design in a diffusional manner)is an advanced algorithm for designing complementary determining regions(CDRs)of the antibody targeted the specific epitope,combining transformer[2]models,3DCNN[3],and diffusion[4]generative models.展开更多
Model-based system-of-systems(SOS)engineering(MBSoSE)is becoming a promising solution for the design of SoS with increasing complexity.However,bridging the models from the design phase to the simulation phase poses si...Model-based system-of-systems(SOS)engineering(MBSoSE)is becoming a promising solution for the design of SoS with increasing complexity.However,bridging the models from the design phase to the simulation phase poses significant challenges and requires an integrated approach.In this study,a unified requirement modeling approach is proposed based on unified architecture framework(UAF).Theoretical models are proposed which compose formalized descriptions from both topdown and bottom-up perspectives.Based on the description,the UAF profile is proposed to represent the SoS mission and constituent systems(CS)goal.Moreover,the agent-based simulation information is also described based on the overview,design concepts,and details(ODD)protocol as the complement part of the SoS profile,which can be transformed into different simulation platforms based on the eXtensible markup language(XML)technology and model-to-text method.In this way,the design of the SoS is simulated automatically in the early design stage.Finally,the method is implemented and an example is given to illustrate the whole process.展开更多
This paper focuses on the question concerning the order of priority between authenticity and inauthenticity in Heidegger’sBeing and Time.Heidegger’s own words suggest sometimes that authenticity precedes inauthentic...This paper focuses on the question concerning the order of priority between authenticity and inauthenticity in Heidegger’sBeing and Time.Heidegger’s own words suggest sometimes that authenticity precedes inauthenticity,sometimes the opposite.This has given rise to a dispute,not only about exegesis,but also about the normative implication of the book.Engaging with interpreters like Dreyfus,Zimmerman,Stambaugh,andGuignon,I argue that it is problematic to assume a distinct mode of“undifferentiatedness”apart from the modes of authenticity and inauthenticity,for the model neither finds textual support nor is systematically tenable.Textual analysis shows that what Heidegger calls a“modal undifferentiatedness”does not itself constitute a distinct undifferentiated mode.Systematic examination shows that the assumption of a distinct mode of undifferentiatedness would oblige the interpreter to assume a“seed”of inauthenticity in this undifferentiatedness,which,once taken seriously,would make superfluous the distinction between undifferentiatedness and inauthent-icity.Moreover,this interpretation leaves incomprehensible how one begins to pursue authenticity.I suggest,instead,a“transformational model”of authenticity,for which philosophical understanding entails a transformation of one’s way of living.To make sense of Heidegger’s seemingly contradictory claims,I distinguish between the existential and the existentiell level of discourse.Authenticity is prior to inauthenticity at the existential level(of understanding),while inauthenticity is prior to authenticity at the existentiell level(of factical existence).I demonstrate that,thanks to a dialogue between these two levels,a global hermeneutic circle is at work in the overall project ofBeing and Time,where authentic living feeds into authentic understanding and vice versa.This makes the tension between authenticity and inauthenticity productive rather than aporetic;it also gives an idea of how the book would‘work’,in an intertwinement of two levels,for a reader as a Dasein.展开更多
Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requir...Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.展开更多
The convolutional neural network(CNN)method based on DeepLabv3+has some problems in the semantic segmentation task of high-resolution remote sensing images,such as fixed receiving field size of feature extraction,lack...The convolutional neural network(CNN)method based on DeepLabv3+has some problems in the semantic segmentation task of high-resolution remote sensing images,such as fixed receiving field size of feature extraction,lack of semantic information,high decoder magnification,and insufficient detail retention ability.A hierarchical feature fusion network(HFFNet)was proposed.Firstly,a combination of transformer and CNN architectures was employed for feature extraction from images of varying resolutions.The extracted features were processed independently.Subsequently,the features from the transformer and CNN were fused under the guidance of features from different sources.This fusion process assisted in restoring information more comprehensively during the decoding stage.Furthermore,a spatial channel attention module was designed in the final stage of decoding to refine features and reduce the semantic gap between shallow CNN features and deep decoder features.The experimental results showed that HFFNet had superior performance on UAVid,LoveDA,Potsdam,and Vaihingen datasets,and its cross-linking index was better than DeepLabv3+and other competing methods,showing strong generalization ability.展开更多
This study proposes a virtual healthcare assistant framework designed to provide support in multiple languages for efficient and accurate healthcare assistance.The system employs a transformer model to process sophist...This study proposes a virtual healthcare assistant framework designed to provide support in multiple languages for efficient and accurate healthcare assistance.The system employs a transformer model to process sophisticated,multilingual user inputs and gain improved contextual understanding compared to conventional models,including long short-term memory(LSTM)models.In contrast to LSTMs,which sequence processes information and may experience challenges with long-range dependencies,transformers utilize self-attention to learn relationships among every aspect of the input in parallel.This enables them to execute more accurately in various languages and contexts,making them well-suited for applications such as translation,summarization,and conversational Comparative evaluations revealed the superiority of the transformer model(accuracy rate:85%)compared with that of the LSTM model(accuracy rate:65%).The experiments revealed several advantages of the transformer architecture over the LSTM model,such as more effective self-attention,the ability for models to work in parallel with each other,and contextual understanding for better multilingual compatibility.Additionally,our prediction model exhibited effectiveness for disease diagnosis,with accuracy of 85%or greater in identifying the relationship between symptoms and diseases among different demographics.The system provides translation support from English to other languages,with conversion to French(Bilingual Evaluation Understudy score:0.7),followed by English to Hindi(0.6).The lowest Bilingual Evaluation Understudy score was found for English to Telugu(0.39).This virtual assistant can also perform symptom analysis and disease prediction,with output given in the preferred language of the user.展开更多
ZIZHITONGJIAN is a key historical work that reflects not only political events but also many culture-loaded expressions rooted in traditional Chinese life.These expressions,including official titles,ritual words,and h...ZIZHITONGJIAN is a key historical work that reflects not only political events but also many culture-loaded expressions rooted in traditional Chinese life.These expressions,including official titles,ritual words,and historical references,carry strong cultural meaning that is hard to translate.And these words are often described as culture-loaded words.Previous research on ZIZHITONGJIAN has offered valuable insights into its translation,focusing on general strategies,historical context,or selected passages.However,these discussions often remain broad in scope,lacking systematic comparison across different types of English editions.This study uses Hu Gengshen’s eco-translatology theory to explore how these culture-loaded words are handled in three kinds of English editions by listing out some classical examples.By applying eco-translatology,this study identifies common translation issues across different English editions and offers a methodological reference for future research on classical Chinese texts,especially in handling culture-loaded words with greater cultural and communicative sensitivity.展开更多
Automated and accurate movie genre classification is crucial for content organization,recommendation systems,and audience targeting in the film industry.Although most existing approaches focus on audiovisual features ...Automated and accurate movie genre classification is crucial for content organization,recommendation systems,and audience targeting in the film industry.Although most existing approaches focus on audiovisual features such as trailers and posters,the text-based classification remains underexplored despite its accessibility and semantic richness.This paper introduces the Genre Attention Model(GAM),a deep learning architecture that integrates transformer models with a hierarchical attention mechanism to extract and leverage contextual information from movie plots formulti-label genre classification.In order to assess its effectiveness,we assessmultiple transformer-based models,including Bidirectional Encoder Representations fromTransformers(BERT),ALite BERT(ALBERT),Distilled BERT(DistilBERT),Robustly Optimized BERT Pretraining Approach(RoBERTa),Efficiently Learning an Encoder that Classifies Token Replacements Accurately(ELECTRA),eXtreme Learning Network(XLNet)and Decodingenhanced BERT with Disentangled Attention(DeBERTa).Experimental results demonstrate the superior performance of DeBERTa-based GAM,which employs a two-tier hierarchical attention mechanism:word-level attention highlights key terms,while sentence-level attention captures critical narrative segments,ensuring a refined and interpretable representation of movie plots.Evaluated on three benchmark datasets Trailers12K,Large Movie Trailer Dataset-9(LMTD-9),and MovieLens37K.GAM achieves micro-average precision scores of 83.63%,83.32%,and 83.34%,respectively,surpassing state-of-the-artmodels.Additionally,GAMis computationally efficient,requiring just 6.10Giga Floating Point Operations Per Second(GFLOPS),making it a scalable and cost-effective solution.These results highlight the growing potential of text-based deep learning models in genre classification and GAM’s effectiveness in improving predictive accuracy while maintaining computational efficiency.With its robust performance,GAM offers a versatile and scalable framework for content recommendation,film indexing,and media analytics,providing an interpretable alternative to traditional audiovisual-based classification techniques.展开更多
The microstructure evolution and bainitic transformation of an Fe-0.19C-4.03Mn-1.48Si steel subjected to near-M_(s)austempering treatment were systematically investigated by combining dilatometer,X-ray diffraction,and...The microstructure evolution and bainitic transformation of an Fe-0.19C-4.03Mn-1.48Si steel subjected to near-M_(s)austempering treatment were systematically investigated by combining dilatometer,X-ray diffraction,and electron microscopy.Three additional austempering treatments with isothermal temperatures above M_(s)were used as benchmarks.Results show that the incubation period for the bainitic transformation occurs when the medium Mn steel is treated with the austempering temperature above M_(s).However,when subjected to near-M_(s)isothermal treatment,the medium Mn steel does not show an incubation period and has the fastest bainitic transformation rate.Moreover,the largest volume fraction of bainite with a value of 74.7%is obtained on the condition of near-M_(s)austempering treatment after cooling to room temperature.Dilatometer and microstructure evolution analysis indicates that the elimination of the incubation period and the fastest rate of bainitic transformation are related to the preformed martensite.The advent of preformed martensite allows the specimen to generate more bainite in a limited time.Considering bainitic ferrite nucleation at austenite grain boundaries and through autocatalysis at ferrite/austenite interfaces,a model is established to understand the kinetics of bainite formation and it can describe the nucleation rate of bainitic transformation well when compared to the experimental results.展开更多
Recent studies employing deep learning to solve the traveling salesman problem(TSP)have mainly focused on learning construction heuristics.Such methods can improve TSP solutions,but still depend on additional programs...Recent studies employing deep learning to solve the traveling salesman problem(TSP)have mainly focused on learning construction heuristics.Such methods can improve TSP solutions,but still depend on additional programs.However,methods that focus on learning improvement heuristics to iteratively refine solutions remain insufficient.Traditional improvement heuristics are guided by a manually designed search strategy and may only achieve limited improvements.This paper proposes a novel framework for learning improvement heuristics,which automatically discovers better improvement policies for heuristics to iteratively solve the TSP.Our framework first designs a new architecture based on a transformer model to make the policy network parameterized,which introduces an action-dropout layer to prevent action selection from overfitting.It then proposes a deep reinforcement learning approach integrating a simulated annealing mechanism(named RL-SA)to learn the pairwise selected policy,aiming to improve the 2-opt algorithm's performance.The RL-SA leverages the whale optimization algorithm to generate initial solutions for better sampling efficiency and uses the Gaussian perturbation strategy to tackle the sparse reward problem of reinforcement learning.The experiment results show that the proposed approach is significantly superior to the state-of-the-art learning-based methods,and further reduces the gap between learning-based methods and highly optimized solvers in the benchmark datasets.Moreover,our pre-trained model M can be applied to guide the SA algorithm(named M-SA(ours)),which performs better than existing deep models in small-,medium-,and large-scale TSPLIB datasets.Additionally,the M-SA(ours)achieves excellent generalization performance in a real-world dataset on global liner shipping routes,with the optimization percentages in distance reduction ranging from3.52%to 17.99%.展开更多
In dynamic 5G network environments,user mobility and heterogeneous network topologies pose dual challenges to the effort of improving performance of mobile edge caching.Existing studies often overlook the dynamic natu...In dynamic 5G network environments,user mobility and heterogeneous network topologies pose dual challenges to the effort of improving performance of mobile edge caching.Existing studies often overlook the dynamic nature of user locations and the potential of device-to-device(D2D)cooperative caching,limiting the reduction of transmission latency.To address this issue,this paper proposes a joint optimization scheme for edge caching that integrates user mobility prediction with deep reinforcement learning.First,a Transformer-based geolocation prediction model is designed,leveraging multi-head attention mechanisms to capture correlations in historical user trajectories for accurate future location prediction.Then,within a three-tier heterogeneous network,we formulate a latency minimization problem under a D2D cooperative caching architecture and develop a mobility-aware Deep Q-Network(DQN)caching strategy.This strategy takes predicted location information as state input and dynamically adjusts the content distribution across small base stations(SBSs)andmobile users(MUs)to reduce end-to-end delay inmulti-hop content retrieval.Simulation results show that the proposed DQN-based method outperforms other baseline strategies across variousmetrics,achieving a 17.2%reduction in transmission delay compared to DQNmethods withoutmobility integration,thus validating the effectiveness of the joint optimization of location prediction and caching decisions.展开更多
The identification of ore grades is a critical step in mineral resource exploration and mining.Prompt gamma neutron activation analysis(PGNAA)technology employs gamma rays generated by the nuclear reactions between ne...The identification of ore grades is a critical step in mineral resource exploration and mining.Prompt gamma neutron activation analysis(PGNAA)technology employs gamma rays generated by the nuclear reactions between neutrons and samples to achieve the qualitative and quantitative detection of sample components.In this study,we present a novel method for identifying copper grade by combining the vision transformer(ViT)model with the PGNAA technique.First,a Monte Carlo simulation is employed to determine the optimal sizes of the neutron moderator,thermal neutron absorption material,and dimensions of the device.Subsequently,based on the parameters obtained through optimization,a PGNAA copper ore measurement model is established.The gamma spectrum of the copper ore is analyzed using the ViT model.The ViT model is optimized for hyperparameters using a grid search.To ensure the reliability of the identification results,the test results are obtained through five repeated tenfold cross-validations.Long short-term memory and convolutional neural network models are compared with the ViT method.These results indicate that the ViT method is efficient in identifying copper ore grades with average accuracy,precision,recall,F_(1)score,and F_(1)(-)score values of 0.9795,0.9637,0.9614,0.9625,and 0.9942,respectively.When identifying associated minerals,the ViT model can identify Pb,Zn,Fe,and Co minerals with identification accuracies of 0.9215,0.9396,0.9966,and 0.8311,respectively.展开更多
Enhancing low-light images with color distortion and uneven multi-light source distribution presents challenges. Most advanced methods for low-light image enhancement are based on the Retinex model using deep learning...Enhancing low-light images with color distortion and uneven multi-light source distribution presents challenges. Most advanced methods for low-light image enhancement are based on the Retinex model using deep learning. Retinexformer introduces channel self-attention mechanisms in the IG-MSA. However, it fails to effectively capture long-range spatial dependencies, leaving room for improvement. Based on the Retinexformer deep learning framework, we designed the Retinexformer+ network. The “+” signifies our advancements in extracting long-range spatial dependencies. We introduced multi-scale dilated convolutions in illumination estimation to expand the receptive field. These convolutions effectively capture the weakening semantic dependency between pixels as distance increases. In illumination restoration, we used Unet++ with multi-level skip connections to better integrate semantic information at different scales. The designed Illumination Fusion Dual Self-Attention (IF-DSA) module embeds multi-scale dilated convolutions to achieve spatial self-attention. This module captures long-range spatial semantic relationships within acceptable computational complexity. Experimental results on the Low-Light (LOL) dataset show that Retexformer+ outperforms other State-Of-The-Art (SOTA) methods in both quantitative and qualitative evaluations, with the computational complexity increased to an acceptable 51.63 G FLOPS. On the LOL_v1 dataset, RetinexFormer+ shows an increase of 1.15 in Peak Signal-to-Noise Ratio (PSNR) and a decrease of 0.39 in Root Mean Square Error (RMSE). On the LOL_v2_real dataset, the PSNR increases by 0.42 and the RMSE decreases by 0.18. Experimental results on the Exdark dataset show that Retexformer+ can effectively enhance real-scene images and maintain their semantic information.展开更多
The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and...The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and insights, influencing daily habits, and driving business, political, and economic decisions. Text posts are particularly significant, and natural language processing (NLP) has emerged as a powerful tool for analyzing such data. While traditional NLP methods have been effective for structured media, social media content poses unique challenges due to its informal and diverse nature. This has spurred the development of new techniques tailored for processing and extracting insights from unstructured user-generated text. One key application of NLP is the summarization of user comments to manage overwhelming content volumes. Abstractive summarization has proven highly effective in generating concise, human-like summaries, offering clear overviews of key themes and sentiments. This enhances understanding and engagement while reducing cognitive effort for users. For businesses, summarization provides actionable insights into customer preferences and feedback, enabling faster trend analysis, improved responsiveness, and strategic adaptability. By distilling complex data into manageable insights, summarization plays a vital role in improving user experiences and empowering informed decision-making in a data-driven landscape. This paper proposes a new implementation framework by fine-tuning and parameterizing Transformer Large Language Models to manage and maintain linguistic and semantic components in abstractive summary generation. The system excels in transforming large volumes of data into meaningful summaries, as evidenced by its strong performance across metrics like fluency, consistency, readability, and semantic coherence.展开更多
High strength steel products with good ductility can be produced via Q&P hot stamping process,while the phase transformation of the process is more complicated than common hot stamping since two-step quenching and...High strength steel products with good ductility can be produced via Q&P hot stamping process,while the phase transformation of the process is more complicated than common hot stamping since two-step quenching and one-step carbon partitioning processes are involved.In this study,an integrated model of microstructure evolution relating to Q&P hot stamping was presented with a persuasively predicted results of mechanical properties.The transformation of diffusional phase and non-diffusional phase,including original austenite grain size individually,were considered,as well as the carbon partitioning process which affects the secondary martensite transformation temperature and the subsequent phase transformations.Afterwards,the mechanical properties including hardness,strength,and elongation were calculated through a series of theoretical and empirical models in accordance with phase contents.Especially,a modified elongation prediction model was generated ultimately with higher accuracy than the existed Mileiko’s model.In the end,the unified model was applied to simulate the Q&P hot stamping process of a U-cup part based on the finite element software LS-DYNA,where the calculated outputs were coincident with the measured consequences.展开更多
Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most ...Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.展开更多
Timed abstract state machine(TASM) is a formal specification language used to specify and simulate the behavior of real-time systems. Formal verification of TASM model can be fulfilled through model checking activitie...Timed abstract state machine(TASM) is a formal specification language used to specify and simulate the behavior of real-time systems. Formal verification of TASM model can be fulfilled through model checking activities by translating into UPPAAL. Firstly, the translational semantics from TASM to UPPAAL is presented through atlas transformation language(ATL). Secondly, the implementation of the proposed model transformation tool TASM2UPPAAL is provided. Finally, a case study is given to illustrate the automatic transformation from TASM model to UPPAAL model.展开更多
Solid-state phase transformation plays an important role in adjusting the microstructure and thus tuning the properties of materials. A general modular, analytical model has been widely applied to describe the kinetic...Solid-state phase transformation plays an important role in adjusting the microstructure and thus tuning the properties of materials. A general modular, analytical model has been widely applied to describe the kinetics of solid-state phase transformation involving nucleation, growth and impingement; the basic conception for iso-kinetics which constitutes a physical foundation for the kinetic models or recipes can be extended by the analytical model. Applying the model, the evolution of kinetic parameters is an effective tool for describing the crystallization of enormous amorphous alloys. In order to further improve the effectiveness of this kinetic model, recently, the recipes and the model fitting procedures were extended, with more factors （e.g., anisotropic growth, soft impingement, and thermodynamic driving force） taken into consideration in the modified models. The recent development in the field of analytical model suggests that it is a general, flexible and open kinetic model for describing the solid-state phase transformation kinetics.展开更多
基金Supported by the National Natural Science Foundation of China(62201293,62034003)the Open-Foundation of State Key Laboratory of Millimeter-Waves(K202313)the Jiangsu Province Youth Science and Technology Talent Support Project(JSTJ-2024-040)。
文摘In this paper,the small-signal modeling of the Indium Phosphide High Electron Mobility Transistor(InP HEMT)based on the Transformer neural network model is investigated.The AC S-parameters of the HEMT device are trained and validated using the Transformer model.In the proposed model,the eight-layer transformer encoders are connected in series and the encoder layer of each Transformer consists of the multi-head attention layer and the feed-forward neural network layer.The experimental results show that the measured and modeled S-parameters of the HEMT device match well in the frequency range of 0.5-40 GHz,with the errors versus frequency less than 1%.Compared with other models,good accuracy can be achieved to verify the effectiveness of the proposed model.
基金funded by Scientific Research Deanship at University of Hail-Saudi Arabia through Project Number RG-23092.
文摘Cyberbullying on social media poses significant psychological risks,yet most detection systems over-simplify the task by focusing on binary classification,ignoring nuanced categories like passive-aggressive remarks or indirect slurs.To address this gap,we propose a hybrid framework combining Term Frequency-Inverse Document Frequency(TF-IDF),word-to-vector(Word2Vec),and Bidirectional Encoder Representations from Transformers(BERT)based models for multi-class cyberbullying detection.Our approach integrates TF-IDF for lexical specificity and Word2Vec for semantic relationships,fused with BERT’s contextual embeddings to capture syntactic and semantic complexities.We evaluate the framework on a publicly available dataset of 47,000 annotated social media posts across five cyberbullying categories:age,ethnicity,gender,religion,and indirect aggression.Among BERT variants tested,BERT Base Un-Cased achieved the highest performance with 93%accuracy(standard deviation across±1%5-fold cross-validation)and an average AUC of 0.96,outperforming standalone TF-IDF(78%)and Word2Vec(82%)models.Notably,it achieved near-perfect AUC scores(0.99)for age and ethnicity-based bullying.A comparative analysis with state-of-the-art benchmarks,including Generative Pre-trained Transformer 2(GPT-2)and Text-to-Text Transfer Transformer(T5)models highlights BERT’s superiority in handling ambiguous language.This work advances cyberbullying detection by demonstrating how hybrid feature extraction and transformer models improve multi-class classification,offering a scalable solution for moderating nuanced harmful content.
基金supported by the Key Project of International Cooperation of Qilu University of Technology(Grant No.:QLUTGJHZ2018008)Shandong Provincial Natural Science Foundation Committee,China(Grant No.:ZR2016HB54)Shandong Provincial Key Laboratory of Microbial Engineering(SME).
文摘AlphaPanda(AlphaFold2[1]inspired protein-specific antibody design in a diffusional manner)is an advanced algorithm for designing complementary determining regions(CDRs)of the antibody targeted the specific epitope,combining transformer[2]models,3DCNN[3],and diffusion[4]generative models.
基金Fifth Electronic Research Institute of the Ministry of Industry and Information Technology(HK07202200877)Pre-research Project on Civil Aerospace Technologies of CNSA(D020101)+2 种基金Zhejiang Provincial Science and Technology Plan Project(2022C01052)Frontier Scientific Research Program of Deep Space Exploration Laboratory(2022-QYKYJHHXYF-018,2022-QYKYJH-GCXD-001)Zhiyuan Laboratory(ZYL2024001)。
文摘Model-based system-of-systems(SOS)engineering(MBSoSE)is becoming a promising solution for the design of SoS with increasing complexity.However,bridging the models from the design phase to the simulation phase poses significant challenges and requires an integrated approach.In this study,a unified requirement modeling approach is proposed based on unified architecture framework(UAF).Theoretical models are proposed which compose formalized descriptions from both topdown and bottom-up perspectives.Based on the description,the UAF profile is proposed to represent the SoS mission and constituent systems(CS)goal.Moreover,the agent-based simulation information is also described based on the overview,design concepts,and details(ODD)protocol as the complement part of the SoS profile,which can be transformed into different simulation platforms based on the eXtensible markup language(XML)technology and model-to-text method.In this way,the design of the SoS is simulated automatically in the early design stage.Finally,the method is implemented and an example is given to illustrate the whole process.
文摘This paper focuses on the question concerning the order of priority between authenticity and inauthenticity in Heidegger’sBeing and Time.Heidegger’s own words suggest sometimes that authenticity precedes inauthenticity,sometimes the opposite.This has given rise to a dispute,not only about exegesis,but also about the normative implication of the book.Engaging with interpreters like Dreyfus,Zimmerman,Stambaugh,andGuignon,I argue that it is problematic to assume a distinct mode of“undifferentiatedness”apart from the modes of authenticity and inauthenticity,for the model neither finds textual support nor is systematically tenable.Textual analysis shows that what Heidegger calls a“modal undifferentiatedness”does not itself constitute a distinct undifferentiated mode.Systematic examination shows that the assumption of a distinct mode of undifferentiatedness would oblige the interpreter to assume a“seed”of inauthenticity in this undifferentiatedness,which,once taken seriously,would make superfluous the distinction between undifferentiatedness and inauthent-icity.Moreover,this interpretation leaves incomprehensible how one begins to pursue authenticity.I suggest,instead,a“transformational model”of authenticity,for which philosophical understanding entails a transformation of one’s way of living.To make sense of Heidegger’s seemingly contradictory claims,I distinguish between the existential and the existentiell level of discourse.Authenticity is prior to inauthenticity at the existential level(of understanding),while inauthenticity is prior to authenticity at the existentiell level(of factical existence).I demonstrate that,thanks to a dialogue between these two levels,a global hermeneutic circle is at work in the overall project ofBeing and Time,where authentic living feeds into authentic understanding and vice versa.This makes the tension between authenticity and inauthenticity productive rather than aporetic;it also gives an idea of how the book would‘work’,in an intertwinement of two levels,for a reader as a Dasein.
文摘Sentence classification is the process of categorizing a sentence based on the context of the sentence.Sentence categorization requires more semantic highlights than other tasks,such as dependence parsing,which requires more syntactic elements.Most existing strategies focus on the general semantics of a conversation without involving the context of the sentence,recognizing the progress and comparing impacts.An ensemble pre-trained language model was taken up here to classify the conversation sentences from the conversation corpus.The conversational sentences are classified into four categories:information,question,directive,and commission.These classification label sequences are for analyzing the conversation progress and predicting the pecking order of the conversation.Ensemble of Bidirectional Encoder for Representation of Transformer(BERT),Robustly Optimized BERT pretraining Approach(RoBERTa),Generative Pre-Trained Transformer(GPT),DistilBERT and Generalized Autoregressive Pretraining for Language Understanding(XLNet)models are trained on conversation corpus with hyperparameters.Hyperparameter tuning approach is carried out for better performance on sentence classification.This Ensemble of Pre-trained Language Models with a Hyperparameter Tuning(EPLM-HT)system is trained on an annotated conversation dataset.The proposed approach outperformed compared to the base BERT,GPT,DistilBERT and XLNet transformer models.The proposed ensemble model with the fine-tuned parameters achieved an F1_score of 0.88.
基金supported by National Natural Science Foundation of China(No.52374155)Anhui Provincial Natural Science Foundation(No.2308085 MF218).
文摘The convolutional neural network(CNN)method based on DeepLabv3+has some problems in the semantic segmentation task of high-resolution remote sensing images,such as fixed receiving field size of feature extraction,lack of semantic information,high decoder magnification,and insufficient detail retention ability.A hierarchical feature fusion network(HFFNet)was proposed.Firstly,a combination of transformer and CNN architectures was employed for feature extraction from images of varying resolutions.The extracted features were processed independently.Subsequently,the features from the transformer and CNN were fused under the guidance of features from different sources.This fusion process assisted in restoring information more comprehensively during the decoding stage.Furthermore,a spatial channel attention module was designed in the final stage of decoding to refine features and reduce the semantic gap between shallow CNN features and deep decoder features.The experimental results showed that HFFNet had superior performance on UAVid,LoveDA,Potsdam,and Vaihingen datasets,and its cross-linking index was better than DeepLabv3+and other competing methods,showing strong generalization ability.
文摘This study proposes a virtual healthcare assistant framework designed to provide support in multiple languages for efficient and accurate healthcare assistance.The system employs a transformer model to process sophisticated,multilingual user inputs and gain improved contextual understanding compared to conventional models,including long short-term memory(LSTM)models.In contrast to LSTMs,which sequence processes information and may experience challenges with long-range dependencies,transformers utilize self-attention to learn relationships among every aspect of the input in parallel.This enables them to execute more accurately in various languages and contexts,making them well-suited for applications such as translation,summarization,and conversational Comparative evaluations revealed the superiority of the transformer model(accuracy rate:85%)compared with that of the LSTM model(accuracy rate:65%).The experiments revealed several advantages of the transformer architecture over the LSTM model,such as more effective self-attention,the ability for models to work in parallel with each other,and contextual understanding for better multilingual compatibility.Additionally,our prediction model exhibited effectiveness for disease diagnosis,with accuracy of 85%or greater in identifying the relationship between symptoms and diseases among different demographics.The system provides translation support from English to other languages,with conversion to French(Bilingual Evaluation Understudy score:0.7),followed by English to Hindi(0.6).The lowest Bilingual Evaluation Understudy score was found for English to Telugu(0.39).This virtual assistant can also perform symptom analysis and disease prediction,with output given in the preferred language of the user.
文摘ZIZHITONGJIAN is a key historical work that reflects not only political events but also many culture-loaded expressions rooted in traditional Chinese life.These expressions,including official titles,ritual words,and historical references,carry strong cultural meaning that is hard to translate.And these words are often described as culture-loaded words.Previous research on ZIZHITONGJIAN has offered valuable insights into its translation,focusing on general strategies,historical context,or selected passages.However,these discussions often remain broad in scope,lacking systematic comparison across different types of English editions.This study uses Hu Gengshen’s eco-translatology theory to explore how these culture-loaded words are handled in three kinds of English editions by listing out some classical examples.By applying eco-translatology,this study identifies common translation issues across different English editions and offers a methodological reference for future research on classical Chinese texts,especially in handling culture-loaded words with greater cultural and communicative sensitivity.
基金would like to thank the Deanship of Graduate Studies and Scientific Research at Qassim University for financial support(QU-APC-2025).
文摘Automated and accurate movie genre classification is crucial for content organization,recommendation systems,and audience targeting in the film industry.Although most existing approaches focus on audiovisual features such as trailers and posters,the text-based classification remains underexplored despite its accessibility and semantic richness.This paper introduces the Genre Attention Model(GAM),a deep learning architecture that integrates transformer models with a hierarchical attention mechanism to extract and leverage contextual information from movie plots formulti-label genre classification.In order to assess its effectiveness,we assessmultiple transformer-based models,including Bidirectional Encoder Representations fromTransformers(BERT),ALite BERT(ALBERT),Distilled BERT(DistilBERT),Robustly Optimized BERT Pretraining Approach(RoBERTa),Efficiently Learning an Encoder that Classifies Token Replacements Accurately(ELECTRA),eXtreme Learning Network(XLNet)and Decodingenhanced BERT with Disentangled Attention(DeBERTa).Experimental results demonstrate the superior performance of DeBERTa-based GAM,which employs a two-tier hierarchical attention mechanism:word-level attention highlights key terms,while sentence-level attention captures critical narrative segments,ensuring a refined and interpretable representation of movie plots.Evaluated on three benchmark datasets Trailers12K,Large Movie Trailer Dataset-9(LMTD-9),and MovieLens37K.GAM achieves micro-average precision scores of 83.63%,83.32%,and 83.34%,respectively,surpassing state-of-the-artmodels.Additionally,GAMis computationally efficient,requiring just 6.10Giga Floating Point Operations Per Second(GFLOPS),making it a scalable and cost-effective solution.These results highlight the growing potential of text-based deep learning models in genre classification and GAM’s effectiveness in improving predictive accuracy while maintaining computational efficiency.With its robust performance,GAM offers a versatile and scalable framework for content recommendation,film indexing,and media analytics,providing an interpretable alternative to traditional audiovisual-based classification techniques.
基金support from the National Natural Science Foundation of China(Grant Nos.52201101 and 52274372)the National Key R&D Program of China(2021YFB3702404)the Fundamental Research Funds for the Central Universities(FRF-TP-22-013A1)。
文摘The microstructure evolution and bainitic transformation of an Fe-0.19C-4.03Mn-1.48Si steel subjected to near-M_(s)austempering treatment were systematically investigated by combining dilatometer,X-ray diffraction,and electron microscopy.Three additional austempering treatments with isothermal temperatures above M_(s)were used as benchmarks.Results show that the incubation period for the bainitic transformation occurs when the medium Mn steel is treated with the austempering temperature above M_(s).However,when subjected to near-M_(s)isothermal treatment,the medium Mn steel does not show an incubation period and has the fastest bainitic transformation rate.Moreover,the largest volume fraction of bainite with a value of 74.7%is obtained on the condition of near-M_(s)austempering treatment after cooling to room temperature.Dilatometer and microstructure evolution analysis indicates that the elimination of the incubation period and the fastest rate of bainitic transformation are related to the preformed martensite.The advent of preformed martensite allows the specimen to generate more bainite in a limited time.Considering bainitic ferrite nucleation at austenite grain boundaries and through autocatalysis at ferrite/austenite interfaces,a model is established to understand the kinetics of bainite formation and it can describe the nucleation rate of bainitic transformation well when compared to the experimental results.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.72101046 and 61672128)。
文摘Recent studies employing deep learning to solve the traveling salesman problem(TSP)have mainly focused on learning construction heuristics.Such methods can improve TSP solutions,but still depend on additional programs.However,methods that focus on learning improvement heuristics to iteratively refine solutions remain insufficient.Traditional improvement heuristics are guided by a manually designed search strategy and may only achieve limited improvements.This paper proposes a novel framework for learning improvement heuristics,which automatically discovers better improvement policies for heuristics to iteratively solve the TSP.Our framework first designs a new architecture based on a transformer model to make the policy network parameterized,which introduces an action-dropout layer to prevent action selection from overfitting.It then proposes a deep reinforcement learning approach integrating a simulated annealing mechanism(named RL-SA)to learn the pairwise selected policy,aiming to improve the 2-opt algorithm's performance.The RL-SA leverages the whale optimization algorithm to generate initial solutions for better sampling efficiency and uses the Gaussian perturbation strategy to tackle the sparse reward problem of reinforcement learning.The experiment results show that the proposed approach is significantly superior to the state-of-the-art learning-based methods,and further reduces the gap between learning-based methods and highly optimized solvers in the benchmark datasets.Moreover,our pre-trained model M can be applied to guide the SA algorithm(named M-SA(ours)),which performs better than existing deep models in small-,medium-,and large-scale TSPLIB datasets.Additionally,the M-SA(ours)achieves excellent generalization performance in a real-world dataset on global liner shipping routes,with the optimization percentages in distance reduction ranging from3.52%to 17.99%.
基金supported by the Liaoning Provincial Education Department Fund,grant number JYTZD2023083.
文摘In dynamic 5G network environments,user mobility and heterogeneous network topologies pose dual challenges to the effort of improving performance of mobile edge caching.Existing studies often overlook the dynamic nature of user locations and the potential of device-to-device(D2D)cooperative caching,limiting the reduction of transmission latency.To address this issue,this paper proposes a joint optimization scheme for edge caching that integrates user mobility prediction with deep reinforcement learning.First,a Transformer-based geolocation prediction model is designed,leveraging multi-head attention mechanisms to capture correlations in historical user trajectories for accurate future location prediction.Then,within a three-tier heterogeneous network,we formulate a latency minimization problem under a D2D cooperative caching architecture and develop a mobility-aware Deep Q-Network(DQN)caching strategy.This strategy takes predicted location information as state input and dynamically adjusts the content distribution across small base stations(SBSs)andmobile users(MUs)to reduce end-to-end delay inmulti-hop content retrieval.Simulation results show that the proposed DQN-based method outperforms other baseline strategies across variousmetrics,achieving a 17.2%reduction in transmission delay compared to DQNmethods withoutmobility integration,thus validating the effectiveness of the joint optimization of location prediction and caching decisions.
基金supported by the National Natural Science Foundation of China(Nos.U2BB2077 and 42374226)the Natural Science Foundation of Jiangxi Province(20232BAB201043 and 20232BCJ23006)the Nuclear energy development project of the National Defense Science and Industry Bureau(Nos.20201192-01,20201192-03).
文摘The identification of ore grades is a critical step in mineral resource exploration and mining.Prompt gamma neutron activation analysis(PGNAA)technology employs gamma rays generated by the nuclear reactions between neutrons and samples to achieve the qualitative and quantitative detection of sample components.In this study,we present a novel method for identifying copper grade by combining the vision transformer(ViT)model with the PGNAA technique.First,a Monte Carlo simulation is employed to determine the optimal sizes of the neutron moderator,thermal neutron absorption material,and dimensions of the device.Subsequently,based on the parameters obtained through optimization,a PGNAA copper ore measurement model is established.The gamma spectrum of the copper ore is analyzed using the ViT model.The ViT model is optimized for hyperparameters using a grid search.To ensure the reliability of the identification results,the test results are obtained through five repeated tenfold cross-validations.Long short-term memory and convolutional neural network models are compared with the ViT method.These results indicate that the ViT method is efficient in identifying copper ore grades with average accuracy,precision,recall,F_(1)score,and F_(1)(-)score values of 0.9795,0.9637,0.9614,0.9625,and 0.9942,respectively.When identifying associated minerals,the ViT model can identify Pb,Zn,Fe,and Co minerals with identification accuracies of 0.9215,0.9396,0.9966,and 0.8311,respectively.
基金supported by the Key Laboratory of Forensic Science and Technology at College of Sichuan Province(2023YB04).
文摘Enhancing low-light images with color distortion and uneven multi-light source distribution presents challenges. Most advanced methods for low-light image enhancement are based on the Retinex model using deep learning. Retinexformer introduces channel self-attention mechanisms in the IG-MSA. However, it fails to effectively capture long-range spatial dependencies, leaving room for improvement. Based on the Retinexformer deep learning framework, we designed the Retinexformer+ network. The “+” signifies our advancements in extracting long-range spatial dependencies. We introduced multi-scale dilated convolutions in illumination estimation to expand the receptive field. These convolutions effectively capture the weakening semantic dependency between pixels as distance increases. In illumination restoration, we used Unet++ with multi-level skip connections to better integrate semantic information at different scales. The designed Illumination Fusion Dual Self-Attention (IF-DSA) module embeds multi-scale dilated convolutions to achieve spatial self-attention. This module captures long-range spatial semantic relationships within acceptable computational complexity. Experimental results on the Low-Light (LOL) dataset show that Retexformer+ outperforms other State-Of-The-Art (SOTA) methods in both quantitative and qualitative evaluations, with the computational complexity increased to an acceptable 51.63 G FLOPS. On the LOL_v1 dataset, RetinexFormer+ shows an increase of 1.15 in Peak Signal-to-Noise Ratio (PSNR) and a decrease of 0.39 in Root Mean Square Error (RMSE). On the LOL_v2_real dataset, the PSNR increases by 0.42 and the RMSE decreases by 0.18. Experimental results on the Exdark dataset show that Retexformer+ can effectively enhance real-scene images and maintain their semantic information.
文摘The rise of social media platforms has revolutionized communication, enabling the exchange of vast amounts of data through text, audio, images, and videos. These platforms have become critical for sharing opinions and insights, influencing daily habits, and driving business, political, and economic decisions. Text posts are particularly significant, and natural language processing (NLP) has emerged as a powerful tool for analyzing such data. While traditional NLP methods have been effective for structured media, social media content poses unique challenges due to its informal and diverse nature. This has spurred the development of new techniques tailored for processing and extracting insights from unstructured user-generated text. One key application of NLP is the summarization of user comments to manage overwhelming content volumes. Abstractive summarization has proven highly effective in generating concise, human-like summaries, offering clear overviews of key themes and sentiments. This enhances understanding and engagement while reducing cognitive effort for users. For businesses, summarization provides actionable insights into customer preferences and feedback, enabling faster trend analysis, improved responsiveness, and strategic adaptability. By distilling complex data into manageable insights, summarization plays a vital role in improving user experiences and empowering informed decision-making in a data-driven landscape. This paper proposes a new implementation framework by fine-tuning and parameterizing Transformer Large Language Models to manage and maintain linguistic and semantic components in abstractive summary generation. The system excels in transforming large volumes of data into meaningful summaries, as evidenced by its strong performance across metrics like fluency, consistency, readability, and semantic coherence.
基金Supported by National Natural Science Foundation of China (Grant Nos. 51775336,U1564203)Program of Shanghai Academic Research Leadership (Grant No. 19XD1401900)
文摘High strength steel products with good ductility can be produced via Q&P hot stamping process,while the phase transformation of the process is more complicated than common hot stamping since two-step quenching and one-step carbon partitioning processes are involved.In this study,an integrated model of microstructure evolution relating to Q&P hot stamping was presented with a persuasively predicted results of mechanical properties.The transformation of diffusional phase and non-diffusional phase,including original austenite grain size individually,were considered,as well as the carbon partitioning process which affects the secondary martensite transformation temperature and the subsequent phase transformations.Afterwards,the mechanical properties including hardness,strength,and elongation were calculated through a series of theoretical and empirical models in accordance with phase contents.Especially,a modified elongation prediction model was generated ultimately with higher accuracy than the existed Mileiko’s model.In the end,the unified model was applied to simulate the Q&P hot stamping process of a U-cup part based on the finite element software LS-DYNA,where the calculated outputs were coincident with the measured consequences.
基金Supported by Shaanxi Province Key Research and Development Project (2021GY-280)the National Natural Science Foundation of China (No.61834005,61772417,61802304)。
文摘Micro-expressions are spontaneous, unconscious movements that reveal true emotions.Accurate facial movement information and network training learning methods are crucial for micro-expression recognition.However, most existing micro-expression recognition technologies so far focus on modeling the single category of micro-expression images and neural network structure.Aiming at the problems of low recognition rate and weak model generalization ability in micro-expression recognition, a micro-expression recognition algorithm is proposed based on graph convolution network(GCN) and Transformer model.Firstly, action unit(AU) feature detection is extracted and facial muscle nodes in the neighborhood are divided into three subsets for recognition.Then, graph convolution layer is used to find the layout of dependencies between AU nodes of micro-expression classification.Finally, multiple attentional features of each facial action are enriched with Transformer model to include more sequence information before calculating the overall correlation of each region.The proposed method is validated in CASME II and CAS(ME)^2 datasets, and the recognition rate reached 69.85%.
基金National Natural Science Foundations of China(No. 61073013,No. 90818024)Aviation Science Foundation of China( No.2010ZAO4001)
文摘Timed abstract state machine(TASM) is a formal specification language used to specify and simulate the behavior of real-time systems. Formal verification of TASM model can be fulfilled through model checking activities by translating into UPPAAL. Firstly, the translational semantics from TASM to UPPAAL is presented through atlas transformation language(ATL). Secondly, the implementation of the proposed model transformation tool TASM2UPPAAL is provided. Finally, a case study is given to illustrate the automatic transformation from TASM model to UPPAAL model.
基金financial support of the National Basic Research Program of China (No. 2011CB610403)the National Natural Science Foundation of China (Nos. 51134011 and 51431008)+1 种基金the Fundamental Research Fund of Northwestern Polytechnical University (No. JC20120223)the China National Funds for Distinguished Young Scientists (No. 51125002)
文摘Solid-state phase transformation plays an important role in adjusting the microstructure and thus tuning the properties of materials. A general modular, analytical model has been widely applied to describe the kinetics of solid-state phase transformation involving nucleation, growth and impingement; the basic conception for iso-kinetics which constitutes a physical foundation for the kinetic models or recipes can be extended by the analytical model. Applying the model, the evolution of kinetic parameters is an effective tool for describing the crystallization of enormous amorphous alloys. In order to further improve the effectiveness of this kinetic model, recently, the recipes and the model fitting procedures were extended, with more factors （e.g., anisotropic growth, soft impingement, and thermodynamic driving force） taken into consideration in the modified models. The recent development in the field of analytical model suggests that it is a general, flexible and open kinetic model for describing the solid-state phase transformation kinetics.
