As the use of deepfake facial videos proliferate,the associated threats to social security and integrity cannot be overstated.Effective methods for detecting forged facial videos are thus urgently needed.While many de...As the use of deepfake facial videos proliferate,the associated threats to social security and integrity cannot be overstated.Effective methods for detecting forged facial videos are thus urgently needed.While many deep learning-based facial forgery detection approaches show promise,they often fail to delve deeply into the complex relationships between image features and forgery indicators,limiting their effectiveness to specific forgery techniques.To address this challenge,we propose a dual-branch collaborative deepfake detection network.The network processes video frame images as input,where a specialized noise extraction module initially extracts the noise feature maps.Subsequently,the original facial images and corresponding noise maps are directed into two parallel feature extraction branches to concurrently learn texture and noise forgery clues.An attention mechanism is employed between the two branches to facilitate mutual guidance and enhancement of texture and noise features across four different scales.This dual-modal feature integration enhances sensitivity to forgery artifacts and boosts generalization ability across various forgery techniques.Features from both branches are then effectively combined and processed through a multi-layer perception layer to distinguish between real and forged video.Experimental results on benchmark deepfake detection datasets demonstrate that our approach outperforms existing state-of-the-art methods in terms of detection performance,accuracy,and generalization ability.展开更多
Since different kinds of face forgeries leave similar forgery traces in videos,learning the common features from different kinds of forged faces would achieve promising generalization ability of forgery detection.Ther...Since different kinds of face forgeries leave similar forgery traces in videos,learning the common features from different kinds of forged faces would achieve promising generalization ability of forgery detection.Therefore,to accurately detect known forgeries while ensuring high generalization ability of detecting unknown forgeries,we propose an intra-inter network(IIN)for face forgery detection(FFD)in videos with continual learning.The proposed IIN mainly consists of three modules,i.e.,intra-module,inter-module,and forged trace masking module(FTMM).Specifically,the intra-module is trained for each kind of face forgeries by supervised learning to extract special features,while the inter-module is trained by self-supervised learning to extract the common features.As a result,the common and special features of the different forgeries are decoupled by the two feature learning modules,and then the decoupled common features can be utlized to achieve high generalization ability for FFD.Moreover,the FTMM is deployed for contrastive learning to further improve detection accuracy.The experimental results on FaceForensic++dataset demonstrate that the proposed IIN outperforms the state-of-the-arts in FFD.Also,the generalization ability of the IIN verified on DFDC and Celeb-DF datasets demonstrates that the proposed IIN significantly improves the generalization ability for FFD.展开更多
Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain f...Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain forgery cues with high transferability.Such cues positively impact the model’s accuracy and generalizability.Moreover,single-modality often causes overfitting of the model,and Red-Green-Blue(RGB)modal-only is not conducive to extracting the more detailed forgery traces.We propose a novel framework for fine-grain forgery cues mining with fusion modality to cope with these issues.First,we propose two functional modules to reveal and locate the deeper forged features.Our method locates deeper forgery cues through a dual-modality progressive fusion module and a noise adaptive enhancement module,which can excavate the association between dualmodal space and channels and enhance the learning of subtle noise features.A sensitive patch branch is introduced on this foundation to enhance the mining of subtle forgery traces under fusion modality.The experimental results demonstrate that our proposed framework can desirably explore the differences between authentic and forged images with supervised learning.Comprehensive evaluations of several mainstream datasets show that our method outperforms the state-of-the-art detection methods with remarkable detection ability and generalizability.展开更多
基金funded by the Ministry of Public Security Science and Technology Program Project(No.2023LL35)the Key Laboratory of Smart Policing and National Security Risk Governance,Sichuan Province(No.ZHZZZD2302).
文摘As the use of deepfake facial videos proliferate,the associated threats to social security and integrity cannot be overstated.Effective methods for detecting forged facial videos are thus urgently needed.While many deep learning-based facial forgery detection approaches show promise,they often fail to delve deeply into the complex relationships between image features and forgery indicators,limiting their effectiveness to specific forgery techniques.To address this challenge,we propose a dual-branch collaborative deepfake detection network.The network processes video frame images as input,where a specialized noise extraction module initially extracts the noise feature maps.Subsequently,the original facial images and corresponding noise maps are directed into two parallel feature extraction branches to concurrently learn texture and noise forgery clues.An attention mechanism is employed between the two branches to facilitate mutual guidance and enhancement of texture and noise features across four different scales.This dual-modal feature integration enhances sensitivity to forgery artifacts and boosts generalization ability across various forgery techniques.Features from both branches are then effectively combined and processed through a multi-layer perception layer to distinguish between real and forged video.Experimental results on benchmark deepfake detection datasets demonstrate that our approach outperforms existing state-of-the-art methods in terms of detection performance,accuracy,and generalization ability.
文摘Since different kinds of face forgeries leave similar forgery traces in videos,learning the common features from different kinds of forged faces would achieve promising generalization ability of forgery detection.Therefore,to accurately detect known forgeries while ensuring high generalization ability of detecting unknown forgeries,we propose an intra-inter network(IIN)for face forgery detection(FFD)in videos with continual learning.The proposed IIN mainly consists of three modules,i.e.,intra-module,inter-module,and forged trace masking module(FTMM).Specifically,the intra-module is trained for each kind of face forgeries by supervised learning to extract special features,while the inter-module is trained by self-supervised learning to extract the common features.As a result,the common and special features of the different forgeries are decoupled by the two feature learning modules,and then the decoupled common features can be utlized to achieve high generalization ability for FFD.Moreover,the FTMM is deployed for contrastive learning to further improve detection accuracy.The experimental results on FaceForensic++dataset demonstrate that the proposed IIN outperforms the state-of-the-arts in FFD.Also,the generalization ability of the IIN verified on DFDC and Celeb-DF datasets demonstrates that the proposed IIN significantly improves the generalization ability for FFD.
基金This study is supported by the Fundamental Research Funds for the Central Universities of PPSUC under Grant 2022JKF02009.
文摘Face forgery detection is drawing ever-increasing attention in the academic community owing to security concerns.Despite the considerable progress in existing methods,we note that:Previous works overlooked finegrain forgery cues with high transferability.Such cues positively impact the model’s accuracy and generalizability.Moreover,single-modality often causes overfitting of the model,and Red-Green-Blue(RGB)modal-only is not conducive to extracting the more detailed forgery traces.We propose a novel framework for fine-grain forgery cues mining with fusion modality to cope with these issues.First,we propose two functional modules to reveal and locate the deeper forged features.Our method locates deeper forgery cues through a dual-modality progressive fusion module and a noise adaptive enhancement module,which can excavate the association between dualmodal space and channels and enhance the learning of subtle noise features.A sensitive patch branch is introduced on this foundation to enhance the mining of subtle forgery traces under fusion modality.The experimental results demonstrate that our proposed framework can desirably explore the differences between authentic and forged images with supervised learning.Comprehensive evaluations of several mainstream datasets show that our method outperforms the state-of-the-art detection methods with remarkable detection ability and generalizability.
