期刊文献+
共找到2篇文章
< 1 >
每页显示 20 50 100
BERT-TECNN模型的文本分类方法研究 被引量:25
1
作者 李铁飞 生龙 吴迪 《计算机工程与应用》 CSCD 北大核心 2021年第18期186-193,共8页
由于Bert-base,Chinese预训练模型参数巨大,在做分类任务微调时内部参数变化较小,易产生过拟合现象,泛化能力弱,且该模型是以字为单位进行的预训练,包含词信息量较少。针对这些问题,提出了BERT-TECNN模型,模型使用Bert-base,Chinese模... 由于Bert-base,Chinese预训练模型参数巨大,在做分类任务微调时内部参数变化较小,易产生过拟合现象,泛化能力弱,且该模型是以字为单位进行的预训练,包含词信息量较少。针对这些问题,提出了BERT-TECNN模型,模型使用Bert-base,Chinese模型作为动态字向量模型,输出包含深度特征信息的字向量,Transformerencoder层再次对数据进行多头自注意力计算,提取特征信息,以提高模型的泛化能力,CNN层利用不同大小卷积核,捕捉每条数据中不同长度词的信息,最后应用softmax进行分类。该模型与Word2Vec+CNN、Word2Vec+BiLSTM、Elmo+CNN、BERT+CNN、BERT+BiLSTM、BERT+Transformer等深度学习文本分类模型在三种数据集上进行对比实验,得到的准确率、精确率、召回率、F1测度值均为最高。实验表明该模型有效地提取了文本中字词的特征信息,优化了过拟合问题,提高了泛化能力。 展开更多
关键词 bert transformer encoder CNN 文本分类 fine-tuning self-attention 过拟合
在线阅读 下载PDF
Vehicle Density Prediction in Low Quality Videos with Transformer Timeseries Prediction Model(TTPM)
2
作者 D.Suvitha M.Vijayalakshmi 《Computer Systems Science & Engineering》 SCIE EI 2023年第1期873-894,共22页
Recent advancement in low-cost cameras has facilitated surveillance in various developing towns in India.The video obtained from such surveillance are of low quality.Still counting vehicles from such videos are necess... Recent advancement in low-cost cameras has facilitated surveillance in various developing towns in India.The video obtained from such surveillance are of low quality.Still counting vehicles from such videos are necessity to avoid traf-fic congestion and allows drivers to plan their routes more precisely.On the other hand,detecting vehicles from such low quality videos are highly challenging with vision based methodologies.In this research a meticulous attempt is made to access low-quality videos to describe traffic in Salem town in India,which is mostly an un-attempted entity by most available sources.In this work profound Detection Transformer(DETR)model is used for object(vehicle)detection.Here vehicles are anticipated in a rush-hour traffic video using a set of loss functions that carry out bipartite coordinating among estimated and information acquired on real attributes.Every frame in the traffic footage has its date and time which is detected and retrieved using Tesseract Optical Character Recognition.The date and time extricated and perceived from the input image are incorporated with the length of the recognized objects acquired from the DETR model.This furnishes the vehicles report with timestamp.Transformer Timeseries Prediction Model(TTPM)is proposed to predict the density of the vehicle for future prediction,here the regular NLP layers have been removed and the encoding temporal layer has been modified.The proposed TTPM error rate outperforms the existing models with RMSE of 4.313 and MAE of 3.812. 展开更多
关键词 Detection transformer self-attention tesseract optical character recognition transformer timeseries prediction model time encoding vector
在线阅读 下载PDF
上一页 1 下一页 到第
使用帮助 返回顶部