Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future...Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future sixth-generation(6G)wireless networks.In this paper,we propose a Chinese text semantic extraction model,namely T-Pointer,to improve the quality of semantic extraction by integrating the Transformer with the pointer-generator network.The proposed T-Pointer model consists of a semantic encoder and a semantic decoder.In the encoding stage,we use the multi-head attention mechanism of the Transformer to extract semantic features from the input Chinese text.In the decoding stage,we first use the Transformer to extract multi-level global text features.Then,we introduce the pointer-generator network model to directly copy the keyword information from the source text.The simulation results demonstrate that the T-Pointer model can improve the bilingual evaluation understudy(BLEU)and recalloriented understudy for gisting evaluation(ROUGE)by 14.69%and 14.87%on average in comparison with the state-of-the-art models,respectively.Also,we implement the T-Pointer model on a semantic communication system based on the universal software radio peripheral(USRP)platform.The result shows that the packet delay of semantic transmission can be reduced by 52.05%on average,compared to traditional information transmission.展开更多
模型可以生成符合用户偏好的摘要.之前的摘要模型侧重于单独控制某个属性,而不是多个属性的组合.传统的Seq2Seq多属性可控文本摘要模型在满足多个控制属性时,存在无法整合所有控制属性、无法准确再现文本中关键信息和无法处理单词表外...模型可以生成符合用户偏好的摘要.之前的摘要模型侧重于单独控制某个属性,而不是多个属性的组合.传统的Seq2Seq多属性可控文本摘要模型在满足多个控制属性时,存在无法整合所有控制属性、无法准确再现文本中关键信息和无法处理单词表外单词等问题.为此,本文提出了一种基于扩展Transformer和指针生成网络(pointer generator network,PGN)的模型.模型中的扩展Transformer将Transformer单编码器-单解码器的模型形式扩展成具有双重文本语义信息提取的双编码器和单个可融合指导信号特征的解码器形式.然后利用指针生成网络模型选择从源文本中复制单词或利用词汇表生成新的摘要信息,以解决摘要任务中常出现的OOV(out of vocabulary)问题.此外,为高效完成位置信息编码,模型在注意力层中使用相对位置表示来引入文本的序列信息.模型可以用于控制摘要的许多重要属性,包括长度、主题和具体性等.通过在公开数据集MACSum上的实验表明,相较以往方法,本文提出的模型在确保摘要质量的同时,更加符合用户给定的属性要求.展开更多
基金National Natural Science Foundation of China under Grants 62122069,62071431,62072490,62301490Science and Technology Development Fund of Macao,Macao,China under Grant 0158/2022/A+2 种基金Guangdong Basic and Applied Basic Research Foundation(2022A1515011287)MYRG2020-00107-IOTSCFDCT SKL-IOTSC(UM)-2021-2023。
文摘Text semantic extraction has been envisioned as a promising solution to improve the data transmission efficiency with the limited radio resources for the autonomous interactions among machines and things in the future sixth-generation(6G)wireless networks.In this paper,we propose a Chinese text semantic extraction model,namely T-Pointer,to improve the quality of semantic extraction by integrating the Transformer with the pointer-generator network.The proposed T-Pointer model consists of a semantic encoder and a semantic decoder.In the encoding stage,we use the multi-head attention mechanism of the Transformer to extract semantic features from the input Chinese text.In the decoding stage,we first use the Transformer to extract multi-level global text features.Then,we introduce the pointer-generator network model to directly copy the keyword information from the source text.The simulation results demonstrate that the T-Pointer model can improve the bilingual evaluation understudy(BLEU)and recalloriented understudy for gisting evaluation(ROUGE)by 14.69%and 14.87%on average in comparison with the state-of-the-art models,respectively.Also,we implement the T-Pointer model on a semantic communication system based on the universal software radio peripheral(USRP)platform.The result shows that the packet delay of semantic transmission can be reduced by 52.05%on average,compared to traditional information transmission.
文摘模型可以生成符合用户偏好的摘要.之前的摘要模型侧重于单独控制某个属性,而不是多个属性的组合.传统的Seq2Seq多属性可控文本摘要模型在满足多个控制属性时,存在无法整合所有控制属性、无法准确再现文本中关键信息和无法处理单词表外单词等问题.为此,本文提出了一种基于扩展Transformer和指针生成网络(pointer generator network,PGN)的模型.模型中的扩展Transformer将Transformer单编码器-单解码器的模型形式扩展成具有双重文本语义信息提取的双编码器和单个可融合指导信号特征的解码器形式.然后利用指针生成网络模型选择从源文本中复制单词或利用词汇表生成新的摘要信息,以解决摘要任务中常出现的OOV(out of vocabulary)问题.此外,为高效完成位置信息编码,模型在注意力层中使用相对位置表示来引入文本的序列信息.模型可以用于控制摘要的许多重要属性,包括长度、主题和具体性等.通过在公开数据集MACSum上的实验表明,相较以往方法,本文提出的模型在确保摘要质量的同时,更加符合用户给定的属性要求.
