In the live broadcast process,eye movement characteristics can reflect people’s attention to the product.However,the existing interest degree predictive model research does not consider the eye movement characteristi...In the live broadcast process,eye movement characteristics can reflect people’s attention to the product.However,the existing interest degree predictive model research does not consider the eye movement characteristics.In order to obtain the users’interest in the product more effectively,we will consider the key eye movement indicators.We first collect eye movement characteristics based on the self-developed data processing algorithm fast discriminative model prediction for tracking(FDIMP),and then we add data dimensions to the original data set through information filling.In addition,we apply the deep factorization machine(DeepFM)architecture to simultaneously learn the combination of low-level and high-level features.In order to effectively learn important features and emphasize relatively important features,the multi-head attention mechanism is applied in the interest model.The experimental results on the public data set Criteo show that,compared with the original DeepFM algorithm,the area under curve(AUC)value was improved by up to 9.32%.展开更多
点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factoriza...点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factorization machine,Self-AtDFEFM)模型。首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测。在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和LogLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验。实验结果表明:在两大数据集上,Self-AtDFEFM模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大。Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性。展开更多
Factorization machine (FM) is an effective model for feature-based recommendation that utilizes inner products to capture second-order feature interactions. However, one of the major drawbacks of FM is that it cannot ...Factorization machine (FM) is an effective model for feature-based recommendation that utilizes inner products to capture second-order feature interactions. However, one of the major drawbacks of FM is that it cannot capture complex high-order interaction signals. A common solution is to change the interaction function, such as stacking deep neural networks on the top level of FM. In this work, we propose an alternative approach to model high-order interaction signals at the embedding level, namely generalized embedding machine (GEM). The embedding used in GEM encodes not only the information from the feature itself but also the information from other correlated features. Under such a situation, the embedding becomes high-order. Then we can incorporate GEM with FM and even its advanced variants to perform feature interactions. More specifically, in this paper, we utilize graph convolution networks (GCN) to generate high-order embeddings. We integrate GEM with several FM-based models and conduct extensive experiments on two real-world datasets. The results demonstrate significant improvement of GEM over the corresponding baselines.展开更多
文摘In the live broadcast process,eye movement characteristics can reflect people’s attention to the product.However,the existing interest degree predictive model research does not consider the eye movement characteristics.In order to obtain the users’interest in the product more effectively,we will consider the key eye movement indicators.We first collect eye movement characteristics based on the self-developed data processing algorithm fast discriminative model prediction for tracking(FDIMP),and then we add data dimensions to the original data set through information filling.In addition,we apply the deep factorization machine(DeepFM)architecture to simultaneously learn the combination of low-level and high-level features.In order to effectively learn important features and emphasize relatively important features,the multi-head attention mechanism is applied in the interest model.The experimental results on the public data set Criteo show that,compared with the original DeepFM algorithm,the area under curve(AUC)value was improved by up to 9.32%.
文摘点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factorization machine,Self-AtDFEFM)模型。首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测。在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和LogLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验。实验结果表明:在两大数据集上,Self-AtDFEFM模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大。Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性。
基金supported by National Natural Science Foundation of China(Nos.62032013 and 61972078)the Fundamental Research Funds for the Central Universities,China(No.N2217004).
文摘Factorization machine (FM) is an effective model for feature-based recommendation that utilizes inner products to capture second-order feature interactions. However, one of the major drawbacks of FM is that it cannot capture complex high-order interaction signals. A common solution is to change the interaction function, such as stacking deep neural networks on the top level of FM. In this work, we propose an alternative approach to model high-order interaction signals at the embedding level, namely generalized embedding machine (GEM). The embedding used in GEM encodes not only the information from the feature itself but also the information from other correlated features. Under such a situation, the embedding becomes high-order. Then we can incorporate GEM with FM and even its advanced variants to perform feature interactions. More specifically, in this paper, we utilize graph convolution networks (GCN) to generate high-order embeddings. We integrate GEM with several FM-based models and conduct extensive experiments on two real-world datasets. The results demonstrate significant improvement of GEM over the corresponding baselines.
