With the widespread use of SMS(Short Message Service),the proliferation of malicious SMS has emerged as a pressing societal issue.While deep learning-based text classifiers offer promise,they often exhibit suboptimal ...With the widespread use of SMS(Short Message Service),the proliferation of malicious SMS has emerged as a pressing societal issue.While deep learning-based text classifiers offer promise,they often exhibit suboptimal performance in fine-grained detection tasks,primarily due to imbalanced datasets and insufficient model representation capabilities.To address this challenge,this paper proposes an LLMs-enhanced graph fusion dual-stream Transformer model for fine-grained Chinese malicious SMS detection.During the data processing stage,Large Language Models(LLMs)are employed for data augmentation,mitigating dataset imbalance.In the data input stage,both word-level and character-level features are utilized as model inputs,enhancing the richness of features and preventing information loss.A dual-stream Transformer serves as the backbone network in the learning representation stage,complemented by a graph-based feature fusion mechanism.At the output stage,both supervised classification cross-entropy loss and supervised contrastive learning loss are used as multi-task optimization objectives,further enhancing the model’s feature representation.Experimental results demonstrate that the proposed method significantly outperforms baselines on a publicly available Chinese malicious SMS dataset.展开更多
SMS spam poses a significant challenge to maintaining user privacy and security.Recently,spammers have employed fraudulent writing styles to bypass spam detection systems.This paper introduces a novel two-level detect...SMS spam poses a significant challenge to maintaining user privacy and security.Recently,spammers have employed fraudulent writing styles to bypass spam detection systems.This paper introduces a novel two-level detection system that utilizes deep learning techniques for effective spam identification to address the challenge of sophisticated SMS spam.The system comprises five steps,beginning with the preprocessing of SMS data.RoBERTa word embedding is then applied to convert text into a numerical format for deep learning analysis.Feature extraction is performed using a Convolutional Neural Network(CNN)for word-level analysis and a Bidirectional Long Short-Term Memory(BiLSTM)for sentence-level analysis.The two-level feature extraction enables a complete understanding of individual words and sentence structure.The novel part of the proposed approach is the Hierarchical Attention Network(HAN),which fuses and selects features at two levels through an attention mechanism.The HAN can deal with words and sentences to focus on the most pertinent aspects of messages for spam detection.This network is productive in capturing meaningful features,considering both word-level and sentence-level semantics.In the classification step,the model classifies the messages into spam and ham.This hybrid deep learning method improve the feature representation,and enhancing the model’s spam detection capabilities.By significantly reducing the incidence of SMS spam,our model contributes to a safer mobile communication environment,protecting users against potential phishing attacks and scams,and aiding in compliance with privacy and security regulations.This model’s performance was evaluated using the SMS Spam Collection Dataset from the UCI Machine Learning Repository.Cross-validation is employed to consider the dataset’s imbalanced nature,ensuring a reliable evaluation.The proposed model achieved a good accuracy of 99.48%,underscoring its efficiency in identifying SMS spam.展开更多
基金supported by the Fundamental Research Funds for the Central Universities(2024JKF13)the Beijing Municipal Education Commission General Program of Science and Technology(No.KM202414019003).
文摘With the widespread use of SMS(Short Message Service),the proliferation of malicious SMS has emerged as a pressing societal issue.While deep learning-based text classifiers offer promise,they often exhibit suboptimal performance in fine-grained detection tasks,primarily due to imbalanced datasets and insufficient model representation capabilities.To address this challenge,this paper proposes an LLMs-enhanced graph fusion dual-stream Transformer model for fine-grained Chinese malicious SMS detection.During the data processing stage,Large Language Models(LLMs)are employed for data augmentation,mitigating dataset imbalance.In the data input stage,both word-level and character-level features are utilized as model inputs,enhancing the richness of features and preventing information loss.A dual-stream Transformer serves as the backbone network in the learning representation stage,complemented by a graph-based feature fusion mechanism.At the output stage,both supervised classification cross-entropy loss and supervised contrastive learning loss are used as multi-task optimization objectives,further enhancing the model’s feature representation.Experimental results demonstrate that the proposed method significantly outperforms baselines on a publicly available Chinese malicious SMS dataset.
文摘SMS spam poses a significant challenge to maintaining user privacy and security.Recently,spammers have employed fraudulent writing styles to bypass spam detection systems.This paper introduces a novel two-level detection system that utilizes deep learning techniques for effective spam identification to address the challenge of sophisticated SMS spam.The system comprises five steps,beginning with the preprocessing of SMS data.RoBERTa word embedding is then applied to convert text into a numerical format for deep learning analysis.Feature extraction is performed using a Convolutional Neural Network(CNN)for word-level analysis and a Bidirectional Long Short-Term Memory(BiLSTM)for sentence-level analysis.The two-level feature extraction enables a complete understanding of individual words and sentence structure.The novel part of the proposed approach is the Hierarchical Attention Network(HAN),which fuses and selects features at two levels through an attention mechanism.The HAN can deal with words and sentences to focus on the most pertinent aspects of messages for spam detection.This network is productive in capturing meaningful features,considering both word-level and sentence-level semantics.In the classification step,the model classifies the messages into spam and ham.This hybrid deep learning method improve the feature representation,and enhancing the model’s spam detection capabilities.By significantly reducing the incidence of SMS spam,our model contributes to a safer mobile communication environment,protecting users against potential phishing attacks and scams,and aiding in compliance with privacy and security regulations.This model’s performance was evaluated using the SMS Spam Collection Dataset from the UCI Machine Learning Repository.Cross-validation is employed to consider the dataset’s imbalanced nature,ensuring a reliable evaluation.The proposed model achieved a good accuracy of 99.48%,underscoring its efficiency in identifying SMS spam.
