为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素...为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素识别的专业性、精准度要求较高等问题,结合自然灾害系统理论的风险要素框架,提出了一种基于双向编码器表征法-双向长短期记忆-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field,BERT-BiLSTM-CRF)的识别方法,并开展了一系列模型验证试验。对比试验结果表明,该模型在准确率、召回率、F_(1)三项指标上均有较好表现,其中准确率为84.62%,召回率为86.19%,F_(1)为85.35%,优于其他对比模型。消融试验结果表明,BERT预训练模型对于该模型性能有着更为显著的影响。综合上述试验结果,可以验证该模型能够有效识别城市内涝舆情信息中的各类风险要素,进而为城市内涝灾害风险管控的数智化转型提供研究依据。展开更多
Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of m...Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of models available to normalize synonymous TCM terms.Therefore,construction of a synonymous term conversion(STC)model for normalizing synonymous TCM terms is necessary.Methods:Based on the neural networks of bidirectional encoder representations from transformers(BERT),four types of TCM STC models were designed:Models based on BERT and text classification,text sequence generation,named entity recognition,and text matching.The superior STC model was selected on the basis of its performance in converting synonymous terms.Moreover,three misjudgment inspection methods for the conversion results of the STC model based on inconsistency were proposed to find incorrect term conversion:Neuron random deactivation,output comparison of multiple isomorphic models,and output comparison of multiple heterogeneous models(OCMH).Results:The classification-based STC model outperformed the other STC task models.It achieved F1 scores of 0.91,0.91,and 0.83 for performing symptoms,patterns,and treatments STC tasks,respectively.The OCMH method showed the best performance in misjudgment inspection,with wrong detection rates of 0.80,0.84,and 0.90 in the term conversion results for symptoms,patterns,and treatments,respectively.Conclusion:The TCM STC model based on classification achieved superior performance in converting synonymous terms for symptoms,patterns,and treatments.The misjudgment inspection method based on OCMH showed superior performance in identifying incorrect outputs.展开更多
Emotion Recognition in Conversations(ERC)is fundamental in creating emotionally intelligentmachines.Graph-BasedNetwork(GBN)models have gained popularity in detecting conversational contexts for ERC tasks.However,their...Emotion Recognition in Conversations(ERC)is fundamental in creating emotionally intelligentmachines.Graph-BasedNetwork(GBN)models have gained popularity in detecting conversational contexts for ERC tasks.However,their limited ability to collect and acquire contextual information hinders their effectiveness.We propose a Text Augmentation-based computational model for recognizing emotions using transformers(TA-MERT)to address this.The proposed model uses the Multimodal Emotion Lines Dataset(MELD),which ensures a balanced representation for recognizing human emotions.Themodel used text augmentation techniques to producemore training data,improving the proposed model’s accuracy.Transformer encoders train the deep neural network(DNN)model,especially Bidirectional Encoder(BE)representations that capture both forward and backward contextual information.This integration improves the accuracy and robustness of the proposed model.Furthermore,we present a method for balancing the training dataset by creating enhanced samples from the original dataset.By balancing the dataset across all emotion categories,we can lessen the adverse effects of data imbalance on the accuracy of the proposed model.Experimental results on the MELD dataset show that TA-MERT outperforms earlier methods,achieving a weighted F1 score of 62.60%and an accuracy of 64.36%.Overall,the proposed TA-MERT model solves the GBN models’weaknesses in obtaining contextual data for ERC.TA-MERT model recognizes human emotions more accurately by employing text augmentation and transformer-based encoding.The balanced dataset and the additional training samples also enhance its resilience.These findings highlight the significance of transformer-based approaches for special emotion recognition in conversations.展开更多
目前在高校C语言编程课程中,使用客观评价的题目难度考验学生的学习情况是非常重要的手段。目前大部分难度评估方法都针对特有科目和特有题型,而对中文编程题目的难度评估存在不足。因此,提出一种融合题目文本和知识点标签的基于BERT(Bi...目前在高校C语言编程课程中,使用客观评价的题目难度考验学生的学习情况是非常重要的手段。目前大部分难度评估方法都针对特有科目和特有题型,而对中文编程题目的难度评估存在不足。因此,提出一种融合题目文本和知识点标签的基于BERT(Bidirectional Encoder Representations from Transformers)和双向长短时记忆(Bi-LSTM)模型的C语言题目难度预测模型FTKB-BiLSTM(Fusion of Title and Knowledge based on BERT and Bi-LSTM)。首先,利用BERT的中文预训练模型获得题目文本和知识点的词向量;其次,融合模块将融合后的信息通过BERT处理得到文本的信息表示,并输入Bi-LSTM模型中学习其中的序列信息,提取更丰富的特征;最后,把经Bi-LSTM模型得到的特征表示通过全连接层并经过Softmax函数处理得到题目难度分类结果。在Leetcode中文数据集和ZjgsuOJ平台数据集上的实验结果表明,相较于XLNet等主流的深度学习模型,所提模型的准确率更优,具有较强的分类能力。展开更多
聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Me...聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Memory,BiLSTM)融合技术的多阶段联合数据治理框架。通过构建有效性判断、语义增强、诉求监测及业务场景分类等核心模块,形成覆盖数据预处理、语义分析、分类预测及诉求应用的全链路治理体系。结果验表明,提出的BERT与BiLSTM融合技术具有较好的性能指标。所提框架通过动态语义特征提取与上下文建模的协同机制,实现客户诉求的细粒度分类和风险点识别,验证基于BERT和BiLSTM的融合模型在电力企业文本类数据处理和应用中的适用性和有效性,为构建自动化数据治理体系提供了更丰富的解决方案。展开更多
针对数控(computer numerical control,CNC)机床故障领域命名实体识别方法中存在实体规范不足及有效实体识别模型缺乏等问题,制定了领域内实体标注策略,提出了一种基于双向转换编码器(bidirectional encoder representations from trans...针对数控(computer numerical control,CNC)机床故障领域命名实体识别方法中存在实体规范不足及有效实体识别模型缺乏等问题,制定了领域内实体标注策略,提出了一种基于双向转换编码器(bidirectional encoder representations from transformers,BERT)的数控机床故障领域命名实体识别方法。采用BERT编码层预训练,将生成向量输入到双向长短期记忆网络(bidirectional long short-term memory,BiLSTM)交互层以提取上下文特征,最终通过条件随机域(conditional random field,CRF)推理层输出预测标签。实验结果表明,BERT-BiLSTM-CRF模型在数控机床故障领域更具优势,与现有模型相比,F_(1)提升大于1.85%。展开更多
命名实体识别属于自然语言处理领域词法分析中的一部分,是计算机正确理解自然语言的基础。为了加强模型对命名实体的识别效果,本文使用预训练模型BERT(bidirectional encoder representation from transformers)作为模型的嵌入层,并针对...命名实体识别属于自然语言处理领域词法分析中的一部分,是计算机正确理解自然语言的基础。为了加强模型对命名实体的识别效果,本文使用预训练模型BERT(bidirectional encoder representation from transformers)作为模型的嵌入层,并针对BERT微调训练对计算机性能要求较高的问题,采用了固定参数嵌入的方式对BERT进行应用,搭建了BERT-BiLSTM-CRF模型。并在该模型的基础上进行了两种改进实验。方法一,继续增加自注意力(self-attention)层,实验结果显示,自注意力层的加入对模型的识别效果提升不明显。方法二,减小BERT模型嵌入层数。实验结果显示,适度减少BERT嵌入层数能够提升模型的命名实体识别准确性,同时又节约了模型的整体训练时间。采用9层嵌入时,在MSRA中文数据集上F1值提升至94.79%,在Weibo中文数据集上F1值达到了68.82%。展开更多
文摘为了实现在城市内涝舆情信息中快速、精准地识别相关风险要素,首先基于新浪微博平台,对用户评论信息及媒体发布信息进行采集、整理及标注,构建了城市内涝灾害事件语料数据集。进而针对城市内涝舆情信息格式不统一、语义复杂且风险要素识别的专业性、精准度要求较高等问题,结合自然灾害系统理论的风险要素框架,提出了一种基于双向编码器表征法-双向长短期记忆-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field,BERT-BiLSTM-CRF)的识别方法,并开展了一系列模型验证试验。对比试验结果表明,该模型在准确率、召回率、F_(1)三项指标上均有较好表现,其中准确率为84.62%,召回率为86.19%,F_(1)为85.35%,优于其他对比模型。消融试验结果表明,BERT预训练模型对于该模型性能有着更为显著的影响。综合上述试验结果,可以验证该模型能够有效识别城市内涝舆情信息中的各类风险要素,进而为城市内涝灾害风险管控的数智化转型提供研究依据。
基金The National Key R&D Program of China supported this study(2017YFC1700303).
文摘Background:The medical records of traditional Chinese medicine(TCM)contain numerous synonymous terms with different descriptions,which is not conducive to computer-aided data mining of TCM.However,there is a lack of models available to normalize synonymous TCM terms.Therefore,construction of a synonymous term conversion(STC)model for normalizing synonymous TCM terms is necessary.Methods:Based on the neural networks of bidirectional encoder representations from transformers(BERT),four types of TCM STC models were designed:Models based on BERT and text classification,text sequence generation,named entity recognition,and text matching.The superior STC model was selected on the basis of its performance in converting synonymous terms.Moreover,three misjudgment inspection methods for the conversion results of the STC model based on inconsistency were proposed to find incorrect term conversion:Neuron random deactivation,output comparison of multiple isomorphic models,and output comparison of multiple heterogeneous models(OCMH).Results:The classification-based STC model outperformed the other STC task models.It achieved F1 scores of 0.91,0.91,and 0.83 for performing symptoms,patterns,and treatments STC tasks,respectively.The OCMH method showed the best performance in misjudgment inspection,with wrong detection rates of 0.80,0.84,and 0.90 in the term conversion results for symptoms,patterns,and treatments,respectively.Conclusion:The TCM STC model based on classification achieved superior performance in converting synonymous terms for symptoms,patterns,and treatments.The misjudgment inspection method based on OCMH showed superior performance in identifying incorrect outputs.
文摘针对现有的中文命名实体识别算法没有充分考虑实体识别任务的数据特征,存在中文样本数据的类别不平衡、训练数据中的噪声太大和每次模型生成数据的分布差异较大的问题,提出了一种以BERT-BiLSTM-CRF(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field)为基线改进的中文命名实体识别模型。首先在BERT-BiLSTM-CRF模型上结合P-Tuning v2技术,精确提取数据特征,然后使用3个损失函数包括聚焦损失(Focal Loss)、标签平滑(Label Smoothing)和KL Loss(Kullback-Leibler divergence loss)作为正则项参与损失计算。实验结果表明,改进的模型在Weibo、Resume和MSRA(Microsoft Research Asia)数据集上的F 1得分分别为71.13%、96.31%、95.90%,验证了所提算法具有更好的性能,并且在不同的下游任务中,所提算法易于与其他的神经网络结合与扩展。
文摘Emotion Recognition in Conversations(ERC)is fundamental in creating emotionally intelligentmachines.Graph-BasedNetwork(GBN)models have gained popularity in detecting conversational contexts for ERC tasks.However,their limited ability to collect and acquire contextual information hinders their effectiveness.We propose a Text Augmentation-based computational model for recognizing emotions using transformers(TA-MERT)to address this.The proposed model uses the Multimodal Emotion Lines Dataset(MELD),which ensures a balanced representation for recognizing human emotions.Themodel used text augmentation techniques to producemore training data,improving the proposed model’s accuracy.Transformer encoders train the deep neural network(DNN)model,especially Bidirectional Encoder(BE)representations that capture both forward and backward contextual information.This integration improves the accuracy and robustness of the proposed model.Furthermore,we present a method for balancing the training dataset by creating enhanced samples from the original dataset.By balancing the dataset across all emotion categories,we can lessen the adverse effects of data imbalance on the accuracy of the proposed model.Experimental results on the MELD dataset show that TA-MERT outperforms earlier methods,achieving a weighted F1 score of 62.60%and an accuracy of 64.36%.Overall,the proposed TA-MERT model solves the GBN models’weaknesses in obtaining contextual data for ERC.TA-MERT model recognizes human emotions more accurately by employing text augmentation and transformer-based encoding.The balanced dataset and the additional training samples also enhance its resilience.These findings highlight the significance of transformer-based approaches for special emotion recognition in conversations.
文摘目前在高校C语言编程课程中,使用客观评价的题目难度考验学生的学习情况是非常重要的手段。目前大部分难度评估方法都针对特有科目和特有题型,而对中文编程题目的难度评估存在不足。因此,提出一种融合题目文本和知识点标签的基于BERT(Bidirectional Encoder Representations from Transformers)和双向长短时记忆(Bi-LSTM)模型的C语言题目难度预测模型FTKB-BiLSTM(Fusion of Title and Knowledge based on BERT and Bi-LSTM)。首先,利用BERT的中文预训练模型获得题目文本和知识点的词向量;其次,融合模块将融合后的信息通过BERT处理得到文本的信息表示,并输入Bi-LSTM模型中学习其中的序列信息,提取更丰富的特征;最后,把经Bi-LSTM模型得到的特征表示通过全连接层并经过Softmax函数处理得到题目难度分类结果。在Leetcode中文数据集和ZjgsuOJ平台数据集上的实验结果表明,相较于XLNet等主流的深度学习模型,所提模型的准确率更优,具有较强的分类能力。
文摘聚焦国家电网客服中心客户诉求数据治理中存在的效率低、人工依赖性强等问题,提出基于Transformer的双向编码器表征(Bidirectional Encoder Representations from Transformer,BERT)和双向长短时记忆(Bi-directional Long Short-Term Memory,BiLSTM)融合技术的多阶段联合数据治理框架。通过构建有效性判断、语义增强、诉求监测及业务场景分类等核心模块,形成覆盖数据预处理、语义分析、分类预测及诉求应用的全链路治理体系。结果验表明,提出的BERT与BiLSTM融合技术具有较好的性能指标。所提框架通过动态语义特征提取与上下文建模的协同机制,实现客户诉求的细粒度分类和风险点识别,验证基于BERT和BiLSTM的融合模型在电力企业文本类数据处理和应用中的适用性和有效性,为构建自动化数据治理体系提供了更丰富的解决方案。
文摘针对数控(computer numerical control,CNC)机床故障领域命名实体识别方法中存在实体规范不足及有效实体识别模型缺乏等问题,制定了领域内实体标注策略,提出了一种基于双向转换编码器(bidirectional encoder representations from transformers,BERT)的数控机床故障领域命名实体识别方法。采用BERT编码层预训练,将生成向量输入到双向长短期记忆网络(bidirectional long short-term memory,BiLSTM)交互层以提取上下文特征,最终通过条件随机域(conditional random field,CRF)推理层输出预测标签。实验结果表明,BERT-BiLSTM-CRF模型在数控机床故障领域更具优势,与现有模型相比,F_(1)提升大于1.85%。
文摘命名实体识别属于自然语言处理领域词法分析中的一部分,是计算机正确理解自然语言的基础。为了加强模型对命名实体的识别效果,本文使用预训练模型BERT(bidirectional encoder representation from transformers)作为模型的嵌入层,并针对BERT微调训练对计算机性能要求较高的问题,采用了固定参数嵌入的方式对BERT进行应用,搭建了BERT-BiLSTM-CRF模型。并在该模型的基础上进行了两种改进实验。方法一,继续增加自注意力(self-attention)层,实验结果显示,自注意力层的加入对模型的识别效果提升不明显。方法二,减小BERT模型嵌入层数。实验结果显示,适度减少BERT嵌入层数能够提升模型的命名实体识别准确性,同时又节约了模型的整体训练时间。采用9层嵌入时,在MSRA中文数据集上F1值提升至94.79%,在Weibo中文数据集上F1值达到了68.82%。
