Recent advances in artificial intelligence and the availability of large-scale benchmarks have made deepfake video generation and manipulation easier.Therefore,developing reliable and robust deepfake video detection m...Recent advances in artificial intelligence and the availability of large-scale benchmarks have made deepfake video generation and manipulation easier.Therefore,developing reliable and robust deepfake video detection mechanisms is paramount.This research introduces a novel real-time deepfake video detection framework by analyzing gaze and blink patterns,addressing the spatial-temporal challenges unique to gaze and blink anomalies using the TimeSformer and hybrid Transformer-CNN models.The TimeSformer architecture leverages spatial-temporal attention mechanisms to capture fine-grained blinking intervals and gaze direction anomalies.Compared to state-of-the-art traditional convolutional models like MesoNet and EfficientNet,which primarily focus on global facial features,our approach emphasizes localized eye-region analysis,significantly enhancing detection accuracy.We evaluate our framework on four standard datasets:FaceForensics,CelebDF-V2,DFDC,and FakeAVCeleb.The proposed framework results reveal higher accuracy,with the TimeSformer model achieving accuracies of 97.5%,96.3%,95.8%,and 97.1%,and with the hybrid Transformer-CNN model demonstrating accuracies of 92.8%,91.5%,90.9%,and 93.2%,on FaceForensics,CelebDF-V2,DFDC,and FakeAVCeleb datasets,respectively,showing robustness in distinguishing manipulated from authentic videos.Our research provides a robust state-of-the-art framework for real-time deepfake video detection.This novel study significantly contributes to video forensics,presenting scalable and accurate real-world application solutions.展开更多
Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme hea...Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme head poses,partial occlusions,and abnormal lighting,remains challenging.Existing models often struggle to effectively focus on discriminative ocular features,leading to suboptimal performance.To address these limitations,this paper proposes dual-branch gaze estimation with Gaussian mixture distribution heatmaps and dynamic adaptive loss function(DMGDL),a novel dual-branch gaze estimation algorithm.By introducing Gaussian mixture distribution heatmaps centered on pupil positions as spatial attention guides,the model is enabled to prioritize ocular regions.Additionally,a dual-branch network architecture is designed to separately extract features for yaw and pitch angles,enhancing flexibility and mitigating cross-angle interference.A dynamic adaptive loss function is further formulated to address discontinuities in angle estimation,improving robustness and convergence stability.Experimental evaluations on three benchmark datasets demonstrate that DMGDL outperforms state-of-the-art methods,achiev-ing a mean angular error of 3.98°on the Max-Planck institute for informatics face gaze(MPI-IFaceGaze)dataset,10.21°on the physically unconstrained gaze estimation in the wild(Gaze360)dataset and 6.14°on the real-time eye gaze estimation in natural environments(RT-Gene)dataset,exhibiting superior generalization and robustness.展开更多
The pandemic situation in 2020 brought about a‘digitized new normal’and created various issues within the current education systems.One of the issues is the monitoring of students during online examination situation...The pandemic situation in 2020 brought about a‘digitized new normal’and created various issues within the current education systems.One of the issues is the monitoring of students during online examination situations.A system to determine the student’s eye gazes during an examination can help to eradicate malpractices.In this work,we track the users’eye gazes by incorporating twelve facial landmarks around both eyes in conjunction with computer vision and the HAAR classifier.We aim to implement eye gaze detection by considering facial landmarks with two different Convolutional Neural Network(CNN)models,namely the AlexNet model and the VGG16 model.The proposed system outperforms the traditional eye gaze detection system which only uses computer vision and the HAAR classifier in several evaluation metric scores.The proposed system is accurate without the need for complex hardware.Therefore,it can be implemented in educational institutes for the fair conduct of examinations,as well as in other instances where eye gaze detection is required.展开更多
A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress ...A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress the direction of the gaze by analyzing images of the eyes,also known as eye patches.However,it is very difficult to construct a person-independent model that can estimate an accurate gaze direction for every person due to individual differences.In this paper,we hypothesize that the difference in the appearance of each of a person’s eyes is related to the difference in the corresponding gaze directions.Based on this hypothesis,a differential eyes’appearances network(DEANet)is trained on public datasets to predict the gaze differences of pairwise eye patches belonging to the same individual.Our proposed DEANet is based on a Siamese neural network(SNNet)framework which has two identical branches.A multi-stream architecture is fed into each branch of the SNNet.Both branches of the DEANet that share the same weights extract the features of the patches;then the features are concatenated to obtain the difference of the gaze directions.Once the differential gaze model is trained,a new person’s gaze direction can be estimated when a few calibrated eye patches for that person are provided.Because personspecific calibrated eye patches are involved in the testing stage,the estimation accuracy is improved.Furthermore,the problem of requiring a large amount of data when training a person-specific model is effectively avoided.A reference grid strategy is also proposed in order to select a few references as some of the DEANet’s inputs directly based on the estimation values,further thereby improving the estimation accuracy.Experiments on public datasets show that our proposed approach outperforms the state-of-theart methods.展开更多
Background Eye tracking te chnology is receiving increased attention in the field of virtual reality.Specifically,future gaze prediction is crucial in pre-computation for many applications such as gaze-contingent rend...Background Eye tracking te chnology is receiving increased attention in the field of virtual reality.Specifically,future gaze prediction is crucial in pre-computation for many applications such as gaze-contingent rendering,advertisement placement,and content-based design.To explore future gaze prediction,it is necessary to analyze the temporal continuity of visual attention in immersive virtual reality.Methods In this paper,the concept of temporal continuity of visual attention is presented.Subsequently,an autocorrelation function method is proposed to evaluate the temporal continuity.Thereafter,the temporal continuity is analyzed in both free-viewing and task-oriented conditions.Results Specifically,in free-viewing conditions,the analysis of a free-viewing gaze dataset indicates that the temporal continuity performs well only within a short time interval.A task-oriented game scene condition was created and conducted to collect users'gaze data.An analysis of the collected gaze data finds the temporal continuity has a similar performance with that of the free-viewing conditions.Temporal continuity can be applied to future gaze prediction and if it is good,users'current gaze positions can be directly utilized to predict their gaze positions in the future.Conclusions The current gaze's future prediction performances are further evaluated in both free-viewing and task-oriented conditions and discover that the current gaze can be efficiently applied to the task of short-term future gaze prediction.The task of long-term gaze prediction still remains to be explored.展开更多
Gaze orientation induces activation of relevant brain regions, presents differences in specificity and time course, and is exhibited in patients with brain injury. However, the components of activated event-related po...Gaze orientation induces activation of relevant brain regions, presents differences in specificity and time course, and is exhibited in patients with brain injury. However, the components of activated event-related potential remain controversial. Previous studies of behavior and cognitive neuroscience related to gaze orientation investigated conscious attention of visual orientation. The present study explored gaze orientation processing-induced event-related potential components and changes with time using reflective orientation of visual attention under a reflective attention cue paradigm. Visual attention processing of gaze orientation was recorded using event-related potential and electroencephalographic recording. Results demonstrated that the reflective attention cue task evoked early directing attention negativity and anterior directing attention negativity, but did not trigger late directing attention positivity. These results suggest that reflective attention occurs over a short time of visual stimulus presentation. During the early stage of attention processing, early directing attention negativity and anterior directing attention negativity were detected, but late directing attention positivity did not occur. These results confirmed reflectivity and time-course superiority of gaze orientation attention processing.展开更多
Prediction of students’engagement in aCollaborative Learning setting is essential to improve the quality of learning.Collaborative learning is a strategy of learning through groups or teams.When cooperative learning ...Prediction of students’engagement in aCollaborative Learning setting is essential to improve the quality of learning.Collaborative learning is a strategy of learning through groups or teams.When cooperative learning behavior occurs,each student in the group should participate in teaching activities.Researchers showed that students who are actively involved in a class gain more.Gaze behavior and facial expression are important nonverbal indicators to reveal engagement in collaborative learning environments.Previous studies require the wearing of sensor devices or eye tracker devices,which have cost barriers and technical interference for daily teaching practice.In this paper,student engagement is automatically analyzed based on computer vision.We tackle the problem of engagement in collaborative learning using a multi-modal deep neural network(MDNN).We combined facial expression and gaze direction as two individual components of MDNN to predict engagement levels in collaborative learning environments.Our multi-modal solution was evaluated in a real collaborative environment.The results show that the model can accurately predict students’performance in the collaborative learning environment.展开更多
In recent years,deep learning techniques have been used to estimate gaze-a significant task in computer vision and human-computer interaction.Previous studies have made significant achievements in predicting 2D or 3D ...In recent years,deep learning techniques have been used to estimate gaze-a significant task in computer vision and human-computer interaction.Previous studies have made significant achievements in predicting 2D or 3D gazes from monocular face images.This study presents a deep neural network for 2D gaze estimation on mobile devices.It achieves state-of-the-art 2D gaze point regression error,while significantly improving gaze classification error on quadrant divisions of the display.To this end,an efficient attention-based module that correlates and fuses the left and right eye contextual features is first proposed to improve gaze point regression performance.Subsequently,through a unified perspective for gaze estimation,metric learning for gaze classification on quadrant divisions is incorporated as additional supervision.Consequently,both gaze point regression and quadrant classification perfor-mances are improved.The experiments demonstrate that the proposed method outperforms existing gaze-estima-tion methods on the GazeCapture and MPIIFaceGaze datasets.展开更多
Gaze information is important for finding region of interest(ROI)which implies where the next action will happen.Supervised gaze estimation does not work on EPIC-Kitchens for lack of ground truth.In this paper,we deve...Gaze information is important for finding region of interest(ROI)which implies where the next action will happen.Supervised gaze estimation does not work on EPIC-Kitchens for lack of ground truth.In this paper,we develop an unsupervised gaze estimation method that helps with egocentric action anticipation.We adopt gaze map as a feature representation,and input it into a multiple modality network jointly with red-green-blue(RGB),optical flow and object features.We explore the method on EGTEA dataset.The estimated gaze map is further optimized with dilation and Gaussian filter,masked onto the original RGB frame and encoded as the important gaze modality.Our results outperform the strong baseline Rolling-Unrolling LSTMs(RULSTM),with top-5 accuracy achieving 34.31%on the seen test set(S1)and 22.07%on unseen test set(S2).The accuracy is improved by 0.58%and 0.87%,respectively.展开更多
We investigated if attentional bias directed to the right increased with age. We assessed the characteristics of the following types of eye-gaze by using the Posner cueing paradigm. Younger (n =16) and older (n = 20) ...We investigated if attentional bias directed to the right increased with age. We assessed the characteristics of the following types of eye-gaze by using the Posner cueing paradigm. Younger (n =16) and older (n = 20) adults participated in this study. First of all, a face which looked straight ahead was presented at the center of screen, followed by a gaze cue that looked left or right. Immediately after this informative cue, a target stimulus (“*”) appeared to the left or right of the face. The stimulus-onset asynchrony (SOA) between the cue and the target was selected from 300, 700, and 1100 ms. Participants were required to judge whether the target appeared to the left or the right of the gaze cue as quickly and accurately as possible. Results showed that older adults indicate a larger positive gaze-cueing effect when the eye-gaze shifted rightward, whereas this effect was not observed for a leftward shift. Moreover, a negative gaze-cueing effect (inhibition of return) was observed when the SOA was longer only for the leftward eye-gaze shift of older adults. These modulations of the cueing effect did not appear in younger adults. These findings demonstrate that the rightward attentional bias in older adults is more robust than the leftward bias.展开更多
This article compares how people with normal bodies and bodies that deviate from dominant media-depicted body ideals, live with and accept their bodies. Media images of ideal bodies encompass judging gazes. These gaze...This article compares how people with normal bodies and bodies that deviate from dominant media-depicted body ideals, live with and accept their bodies. Media images of ideal bodies encompass judging gazes. These gazes affect and discipline people and may make it challenging for them to accept their bodies. The data material is part of the interdisciplinary Nordic project called “Beauty comes from within: looking good as a challenge in health promotion”. Based on 20 interviews with Norwegian men and women, of whom 10 have particular appearance-related problems, the article discusses the relationship between the media-depicted body ideals, descriptions by informants of what a good-looking body is, body satisfaction and body practices. The article shows resonance between how people describe good-looking bodies and satisfaction or not with own bodies. Women express more dissatisfaction with their bodies than men, but the article shows that many have strategies for trying to accept their bodies as they are. The comparative perspective highlight that the people having deviant bodies, more than those with normal bodies, balance the idea of “being myself” with the idea of “doing the best out of my (bodily) situation”. Most interestingly, they show that it is harder to accept handicaps that are changeable, like overweight, than harelips, deformed legs and skin injuries. As such, overweight becomes a double burden.展开更多
This paper focuses the question： What does it mean to be a traveller rather than a tourist？ The term “tourism” ismostlyused in impersonal commercial language but “travel” often implies the personal, picaresque s...This paper focuses the question： What does it mean to be a traveller rather than a tourist？ The term “tourism” ismostlyused in impersonal commercial language but “travel” often implies the personal, picaresque style of travel writing. The travellerbeing the hero of the text and the tourist as an unfortunate by-product of globalisation highlight the formation of the important binary opposites through the identity/difference logic.Travel writers deprecate the behaviour of tourists and go for a more authentic way to engage with cultural contrastfor a more concrete example of otherness. The primary texts taken for this study are the select Odia travel writers： GobindaDas’sDese Dese （In Countries）, GolakbihariDhal’sLondon Chithi （Letter From London）, and Pratibha Ray’s Swapnara Alaska （Dreamy Alaska） and Africa NayikaNilanadi （Africa’s Heroine the River Nile）.展开更多
Skin manifestations can be major sources of stress for patients with skin diseases;hence, the effective use of makeup and cosmetic products for these patients has been established. The objective of this study was to d...Skin manifestations can be major sources of stress for patients with skin diseases;hence, the effective use of makeup and cosmetic products for these patients has been established. The objective of this study was to determine if makeup can divert observers’ gaze from areas of inflammatory acne lesions. Both base and point makeup were applied to two Japanese female patients with mild to moderate acne vulgaris to hide skin manifestations, as well as to accentuate the eyes and lips. Photographs of their faces were shown, at various stages of makeup application, to 22 observers (11 men and 11 women). The effects of makeup application, and other eye-diverting strategies (e.g., clothing, accessories, and hairstyle), used to draw observers’ gaze away from acne lesions, were evaluated by analyzing observers’ eye movements. As base makeup application proceeded, time to first fixation, total fixation duration, and fixation count changed. Compared to “no makeup”, the time to first fixation, total fixation duration, and fixation count also decreased significantly after point makeup application. The additional eye-diverting strategies used also had significant gaze-diverting effects. Therefore, makeup can be useful for patients with acne to divert others’ gaze from lesions. Therefore, it should be actively integrated into acne management.展开更多
Much more than simple viewing, gaze, which is a kind of concentrated thorough long-term viewing, makes the gazer and the gazed establish a complicated power relationship. Frequently, women are in a position of being g...Much more than simple viewing, gaze, which is a kind of concentrated thorough long-term viewing, makes the gazer and the gazed establish a complicated power relationship. Frequently, women are in a position of being gazed at by others instead of taking the initiative to have a counter gaze. Consequently, they are confronted with abnormal self identity construction and alienation under the gazes of others and the self gaze brought by others’ gazes. This is fully revealed in the experiences of three representative female characters in Fitzgerald’s The Great Gatsby. This paper elaborates on their experiences and the relative consequences under gazes-deviation of femininity and other aspects, in the hope of providing a refreshing perspective for the study on this novel, gaze and gender issues.展开更多
Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination...Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination conditions,real-world scenarios continue to provide difficulties for accurate and consistent eye gaze recognition.This work is aimed at investigating the potential benefits of employing transfer learning to improve eye gaze detection ability and efficiency.Transfer learning is the process of fine-tuning pre-trained models on smaller,domain-specific datasets after they have been trained on larger datasets.We study several transfer learning algorithms and evaluate their effectiveness on eye gaze identification,including both Regression and Classification tasks,using a range of deep learning architectures,namely AlexNet,Visual Geometry Group(VGG),InceptionV3,and ResNet.In this study,we evaluate the effectiveness of transfer learning-basedmodels against models that were trained fromscratch using eye-gazing datasets on grounds of various performance and loss metrics such as Precision,Accuracy,and Mean Absolute Error.We investigate the effects of different pre-trainedmodels,dataset sizes,and domain gaps on the transfer learning process,and the findings of our study clarify the efficacy of transfer learning for eye gaze detection and offer suggestions for the most successful transfer learning strategies to apply in real-world situations.展开更多
Throughout the lifespan,an animal can encounter predators frequently,thus the ability to avoid attacks from predators is crucial for its survival.The chances of evading danger can be greatly improved if the animal can...Throughout the lifespan,an animal can encounter predators frequently,thus the ability to avoid attacks from predators is crucial for its survival.The chances of evading danger can be greatly improved if the animal can respond immediately to the threat.Therefore,when an animal detects a threat through its visual system,it must quickly direct its gaze and attention toward the source of danger,assess the threat level,and take appropriate action.展开更多
This paper probes into the transformation process and the deep meaning of the female characters in The Case of a Missing Seventeen and Gone Girl.The research shows that Wang Di and Amy realize the transformation from...This paper probes into the transformation process and the deep meaning of the female characters in The Case of a Missing Seventeen and Gone Girl.The research shows that Wang Di and Amy realize the transformation from“being gazed at”to“gazer”through the disappearance in the Chinese and Western cultural backgrounds respectively.Wang Di’s secret resistance reflects the traditional culture’s suppression of women’s self-awareness,while Amy controls the development of the story through carefully planned strategies,revealing the complex position of women in the marriage system.These two works show the struggle and breakthrough of women in the gender power structure,and emphasize the necessity of gender equality consciousness.Through a detailed study of role analysis and narrative strategies,this paper provides a new perspective for understanding the survival and growth of modern women in gender relations.展开更多
The biathlon,an Olympic sporting discipline that combines cross-country skiing with rifle marksmanship,entails considerable physiological demands,as well as fine motor control while shooting after intense exercise and...The biathlon,an Olympic sporting discipline that combines cross-country skiing with rifle marksmanship,entails considerable physiological demands,as well as fine motor control while shooting after intense exercise and under mental pressure.Although much of our knowledge about cross-country skiing is probably also applicable to the biathlon,carrying the rifle and shooting under stress make this discipline somewhat unique.The present review summarizes and examines the scientific literature related to biathlon performance,with a focus on physiological and biomechanical factors and shooting technique,as well as psychophysiological aspects of shooting performance.We conclude with suggestions for future research designed to extend our knowledge about the biathlon,which is presently quite limited.2018 Published by Elsevier B.V.on behalf of Shanghai University of Sport.This is an open access article under the CC BY-NC-ND license.(http://creativecommons.org/licenses/by-nc-nd/4.0/).展开更多
Eye center localization is one of the most crucial and basic requirements for some human-computer interaction applications such as eye gaze estimation and eye tracking. There is a large body of works on this topic in ...Eye center localization is one of the most crucial and basic requirements for some human-computer interaction applications such as eye gaze estimation and eye tracking. There is a large body of works on this topic in recent years, but the accuracy still needs to be improved due to challenges in appearance such as the high variability of shapes, lighting conditions, viewing angles and possible occlusions. To address these problems and limitations, we propose a novel approach in this paper for the eye center localization with a fully convolutional network(FCN),which is an end-to-end and pixels-to-pixels network and can locate the eye center accurately. The key idea is to apply the FCN from the object semantic segmentation task to the eye center localization task since the problem of eye center localization can be regarded as a special semantic segmentation problem. We adapt contemporary FCN into a shallow structure with a large kernel convolutional block and transfer their performance from semantic segmentation to the eye center localization task by fine-tuning.Extensive experiments show that the proposed method outperforms the state-of-the-art methods in both accuracy and reliability of eye center localization. The proposed method has achieved a large performance improvement on the most challenging database and it thus provides a promising solution to some challenging applications.展开更多
An active stereo vision system based on a model of neural pathways of human binocular motor system is proposed. With this model, it is guaranteed that the two cameras of the active stereo vision system can keep their ...An active stereo vision system based on a model of neural pathways of human binocular motor system is proposed. With this model, it is guaranteed that the two cameras of the active stereo vision system can keep their lines of sight fixed on the same target object during smooth pursuit. This feature is very important for active stereo vision systems, since not only 3D reconstruction needs the two cameras have an overlapping field of vision, but also it can facilitate the 3D reconstruction algorithm. To evaluate the effectiveness of the proposed method, some software simulations are done to demonstrate the same target tracking characteristic in a virtual environment apt to mistracking easily. Here, mistracking means two eyes track two different objects separately. Then the proposed method is implemented in our active stereo vision system to perform real tracking task in a laboratory scene where several persons walk self-determining. Before the proposed model is implemented in the system, mistracking occurred frequently. After it is enabled, mistracking never occurred. The result shows that the vision system based on neural pathways of human binocular motor system can reliably avoid mistracking.展开更多
文摘Recent advances in artificial intelligence and the availability of large-scale benchmarks have made deepfake video generation and manipulation easier.Therefore,developing reliable and robust deepfake video detection mechanisms is paramount.This research introduces a novel real-time deepfake video detection framework by analyzing gaze and blink patterns,addressing the spatial-temporal challenges unique to gaze and blink anomalies using the TimeSformer and hybrid Transformer-CNN models.The TimeSformer architecture leverages spatial-temporal attention mechanisms to capture fine-grained blinking intervals and gaze direction anomalies.Compared to state-of-the-art traditional convolutional models like MesoNet and EfficientNet,which primarily focus on global facial features,our approach emphasizes localized eye-region analysis,significantly enhancing detection accuracy.We evaluate our framework on four standard datasets:FaceForensics,CelebDF-V2,DFDC,and FakeAVCeleb.The proposed framework results reveal higher accuracy,with the TimeSformer model achieving accuracies of 97.5%,96.3%,95.8%,and 97.1%,and with the hybrid Transformer-CNN model demonstrating accuracies of 92.8%,91.5%,90.9%,and 93.2%,on FaceForensics,CelebDF-V2,DFDC,and FakeAVCeleb datasets,respectively,showing robustness in distinguishing manipulated from authentic videos.Our research provides a robust state-of-the-art framework for real-time deepfake video detection.This novel study significantly contributes to video forensics,presenting scalable and accurate real-world application solutions.
基金supported by the Key Project of the NationalLanguage Commission(No.ZDI145-110)the AcademicResearch Projects of Beijing Union University(No.ZK20202514)+1 种基金the Key Laboratory Project(No.YYZN-2024-6)the Project for the Construction and Support of High-Level Innovative Teams in Beijing Municipal Institutions(No.BPHR20220121).
文摘Gaze estimation,a crucial non-verbal communication cue,has achieved remarkable progress through convolutional neural networks.However,accurate gaze prediction in uncon-strained environments,particularly in extreme head poses,partial occlusions,and abnormal lighting,remains challenging.Existing models often struggle to effectively focus on discriminative ocular features,leading to suboptimal performance.To address these limitations,this paper proposes dual-branch gaze estimation with Gaussian mixture distribution heatmaps and dynamic adaptive loss function(DMGDL),a novel dual-branch gaze estimation algorithm.By introducing Gaussian mixture distribution heatmaps centered on pupil positions as spatial attention guides,the model is enabled to prioritize ocular regions.Additionally,a dual-branch network architecture is designed to separately extract features for yaw and pitch angles,enhancing flexibility and mitigating cross-angle interference.A dynamic adaptive loss function is further formulated to address discontinuities in angle estimation,improving robustness and convergence stability.Experimental evaluations on three benchmark datasets demonstrate that DMGDL outperforms state-of-the-art methods,achiev-ing a mean angular error of 3.98°on the Max-Planck institute for informatics face gaze(MPI-IFaceGaze)dataset,10.21°on the physically unconstrained gaze estimation in the wild(Gaze360)dataset and 6.14°on the real-time eye gaze estimation in natural environments(RT-Gene)dataset,exhibiting superior generalization and robustness.
基金funded by the“Intelligent Recognition Industry Service Research Center”from The Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education(MOE)in Taiwan.Grant Number:N/A and the APC was funded by the aforementioned Project.
文摘The pandemic situation in 2020 brought about a‘digitized new normal’and created various issues within the current education systems.One of the issues is the monitoring of students during online examination situations.A system to determine the student’s eye gazes during an examination can help to eradicate malpractices.In this work,we track the users’eye gazes by incorporating twelve facial landmarks around both eyes in conjunction with computer vision and the HAAR classifier.We aim to implement eye gaze detection by considering facial landmarks with two different Convolutional Neural Network(CNN)models,namely the AlexNet model and the VGG16 model.The proposed system outperforms the traditional eye gaze detection system which only uses computer vision and the HAAR classifier in several evaluation metric scores.The proposed system is accurate without the need for complex hardware.Therefore,it can be implemented in educational institutes for the fair conduct of examinations,as well as in other instances where eye gaze detection is required.
基金supported by the Science and Technology Support Project of Sichuan Science and Technology Department(2018SZ0357)and China Scholarship。
文摘A person’s eye gaze can effectively express that person’s intentions.Thus,gaze estimation is an important approach in intelligent manufacturing to analyze a person’s intentions.Many gaze estimation methods regress the direction of the gaze by analyzing images of the eyes,also known as eye patches.However,it is very difficult to construct a person-independent model that can estimate an accurate gaze direction for every person due to individual differences.In this paper,we hypothesize that the difference in the appearance of each of a person’s eyes is related to the difference in the corresponding gaze directions.Based on this hypothesis,a differential eyes’appearances network(DEANet)is trained on public datasets to predict the gaze differences of pairwise eye patches belonging to the same individual.Our proposed DEANet is based on a Siamese neural network(SNNet)framework which has two identical branches.A multi-stream architecture is fed into each branch of the SNNet.Both branches of the DEANet that share the same weights extract the features of the patches;then the features are concatenated to obtain the difference of the gaze directions.Once the differential gaze model is trained,a new person’s gaze direction can be estimated when a few calibrated eye patches for that person are provided.Because personspecific calibrated eye patches are involved in the testing stage,the estimation accuracy is improved.Furthermore,the problem of requiring a large amount of data when training a person-specific model is effectively avoided.A reference grid strategy is also proposed in order to select a few references as some of the DEANet’s inputs directly based on the estimation values,further thereby improving the estimation accuracy.Experiments on public datasets show that our proposed approach outperforms the state-of-theart methods.
基金the National Key R&D Program of China(2017 YFB 0203000)National Natural Science Foundation of China(61632003,61661146002,61631001).
文摘Background Eye tracking te chnology is receiving increased attention in the field of virtual reality.Specifically,future gaze prediction is crucial in pre-computation for many applications such as gaze-contingent rendering,advertisement placement,and content-based design.To explore future gaze prediction,it is necessary to analyze the temporal continuity of visual attention in immersive virtual reality.Methods In this paper,the concept of temporal continuity of visual attention is presented.Subsequently,an autocorrelation function method is proposed to evaluate the temporal continuity.Thereafter,the temporal continuity is analyzed in both free-viewing and task-oriented conditions.Results Specifically,in free-viewing conditions,the analysis of a free-viewing gaze dataset indicates that the temporal continuity performs well only within a short time interval.A task-oriented game scene condition was created and conducted to collect users'gaze data.An analysis of the collected gaze data finds the temporal continuity has a similar performance with that of the free-viewing conditions.Temporal continuity can be applied to future gaze prediction and if it is good,users'current gaze positions can be directly utilized to predict their gaze positions in the future.Conclusions The current gaze's future prediction performances are further evaluated in both free-viewing and task-oriented conditions and discover that the current gaze can be efficiently applied to the task of short-term future gaze prediction.The task of long-term gaze prediction still remains to be explored.
基金the Applied Experimental Psychology Project of Beijing Key Laboratory from 2008 to 2009, No.JD100270541the Scientific Research Foundation of Beijing Normal University, No.2009SC-3
文摘Gaze orientation induces activation of relevant brain regions, presents differences in specificity and time course, and is exhibited in patients with brain injury. However, the components of activated event-related potential remain controversial. Previous studies of behavior and cognitive neuroscience related to gaze orientation investigated conscious attention of visual orientation. The present study explored gaze orientation processing-induced event-related potential components and changes with time using reflective orientation of visual attention under a reflective attention cue paradigm. Visual attention processing of gaze orientation was recorded using event-related potential and electroencephalographic recording. Results demonstrated that the reflective attention cue task evoked early directing attention negativity and anterior directing attention negativity, but did not trigger late directing attention positivity. These results suggest that reflective attention occurs over a short time of visual stimulus presentation. During the early stage of attention processing, early directing attention negativity and anterior directing attention negativity were detected, but late directing attention positivity did not occur. These results confirmed reflectivity and time-course superiority of gaze orientation attention processing.
基金supported by the National Natural Science Foundation of China (No.61977031)XPCC’s Plan for Tackling Key Scientific and Technological Problems in Key Fields (No.2021AB023-3).
文摘Prediction of students’engagement in aCollaborative Learning setting is essential to improve the quality of learning.Collaborative learning is a strategy of learning through groups or teams.When cooperative learning behavior occurs,each student in the group should participate in teaching activities.Researchers showed that students who are actively involved in a class gain more.Gaze behavior and facial expression are important nonverbal indicators to reveal engagement in collaborative learning environments.Previous studies require the wearing of sensor devices or eye tracker devices,which have cost barriers and technical interference for daily teaching practice.In this paper,student engagement is automatically analyzed based on computer vision.We tackle the problem of engagement in collaborative learning using a multi-modal deep neural network(MDNN).We combined facial expression and gaze direction as two individual components of MDNN to predict engagement levels in collaborative learning environments.Our multi-modal solution was evaluated in a real collaborative environment.The results show that the model can accurately predict students’performance in the collaborative learning environment.
基金the National Natural Science Foundation of China,No.61932003and the Fundamental Research Funds for the Central Universities.
文摘In recent years,deep learning techniques have been used to estimate gaze-a significant task in computer vision and human-computer interaction.Previous studies have made significant achievements in predicting 2D or 3D gazes from monocular face images.This study presents a deep neural network for 2D gaze estimation on mobile devices.It achieves state-of-the-art 2D gaze point regression error,while significantly improving gaze classification error on quadrant divisions of the display.To this end,an efficient attention-based module that correlates and fuses the left and right eye contextual features is first proposed to improve gaze point regression performance.Subsequently,through a unified perspective for gaze estimation,metric learning for gaze classification on quadrant divisions is incorporated as additional supervision.Consequently,both gaze point regression and quadrant classification perfor-mances are improved.The experiments demonstrate that the proposed method outperforms existing gaze-estima-tion methods on the GazeCapture and MPIIFaceGaze datasets.
基金Supported by the National Natural Science Foundation of China(61772328)
文摘Gaze information is important for finding region of interest(ROI)which implies where the next action will happen.Supervised gaze estimation does not work on EPIC-Kitchens for lack of ground truth.In this paper,we develop an unsupervised gaze estimation method that helps with egocentric action anticipation.We adopt gaze map as a feature representation,and input it into a multiple modality network jointly with red-green-blue(RGB),optical flow and object features.We explore the method on EGTEA dataset.The estimated gaze map is further optimized with dilation and Gaussian filter,masked onto the original RGB frame and encoded as the important gaze modality.Our results outperform the strong baseline Rolling-Unrolling LSTMs(RULSTM),with top-5 accuracy achieving 34.31%on the seen test set(S1)and 22.07%on unseen test set(S2).The accuracy is improved by 0.58%and 0.87%,respectively.
文摘We investigated if attentional bias directed to the right increased with age. We assessed the characteristics of the following types of eye-gaze by using the Posner cueing paradigm. Younger (n =16) and older (n = 20) adults participated in this study. First of all, a face which looked straight ahead was presented at the center of screen, followed by a gaze cue that looked left or right. Immediately after this informative cue, a target stimulus (“*”) appeared to the left or right of the face. The stimulus-onset asynchrony (SOA) between the cue and the target was selected from 300, 700, and 1100 ms. Participants were required to judge whether the target appeared to the left or the right of the gaze cue as quickly and accurately as possible. Results showed that older adults indicate a larger positive gaze-cueing effect when the eye-gaze shifted rightward, whereas this effect was not observed for a leftward shift. Moreover, a negative gaze-cueing effect (inhibition of return) was observed when the SOA was longer only for the leftward eye-gaze shift of older adults. These modulations of the cueing effect did not appear in younger adults. These findings demonstrate that the rightward attentional bias in older adults is more robust than the leftward bias.
文摘This article compares how people with normal bodies and bodies that deviate from dominant media-depicted body ideals, live with and accept their bodies. Media images of ideal bodies encompass judging gazes. These gazes affect and discipline people and may make it challenging for them to accept their bodies. The data material is part of the interdisciplinary Nordic project called “Beauty comes from within: looking good as a challenge in health promotion”. Based on 20 interviews with Norwegian men and women, of whom 10 have particular appearance-related problems, the article discusses the relationship between the media-depicted body ideals, descriptions by informants of what a good-looking body is, body satisfaction and body practices. The article shows resonance between how people describe good-looking bodies and satisfaction or not with own bodies. Women express more dissatisfaction with their bodies than men, but the article shows that many have strategies for trying to accept their bodies as they are. The comparative perspective highlight that the people having deviant bodies, more than those with normal bodies, balance the idea of “being myself” with the idea of “doing the best out of my (bodily) situation”. Most interestingly, they show that it is harder to accept handicaps that are changeable, like overweight, than harelips, deformed legs and skin injuries. As such, overweight becomes a double burden.
文摘This paper focuses the question： What does it mean to be a traveller rather than a tourist？ The term “tourism” ismostlyused in impersonal commercial language but “travel” often implies the personal, picaresque style of travel writing. The travellerbeing the hero of the text and the tourist as an unfortunate by-product of globalisation highlight the formation of the important binary opposites through the identity/difference logic.Travel writers deprecate the behaviour of tourists and go for a more authentic way to engage with cultural contrastfor a more concrete example of otherness. The primary texts taken for this study are the select Odia travel writers： GobindaDas’sDese Dese （In Countries）, GolakbihariDhal’sLondon Chithi （Letter From London）, and Pratibha Ray’s Swapnara Alaska （Dreamy Alaska） and Africa NayikaNilanadi （Africa’s Heroine the River Nile）.
文摘Skin manifestations can be major sources of stress for patients with skin diseases;hence, the effective use of makeup and cosmetic products for these patients has been established. The objective of this study was to determine if makeup can divert observers’ gaze from areas of inflammatory acne lesions. Both base and point makeup were applied to two Japanese female patients with mild to moderate acne vulgaris to hide skin manifestations, as well as to accentuate the eyes and lips. Photographs of their faces were shown, at various stages of makeup application, to 22 observers (11 men and 11 women). The effects of makeup application, and other eye-diverting strategies (e.g., clothing, accessories, and hairstyle), used to draw observers’ gaze away from acne lesions, were evaluated by analyzing observers’ eye movements. As base makeup application proceeded, time to first fixation, total fixation duration, and fixation count changed. Compared to “no makeup”, the time to first fixation, total fixation duration, and fixation count also decreased significantly after point makeup application. The additional eye-diverting strategies used also had significant gaze-diverting effects. Therefore, makeup can be useful for patients with acne to divert others’ gaze from lesions. Therefore, it should be actively integrated into acne management.
文摘Much more than simple viewing, gaze, which is a kind of concentrated thorough long-term viewing, makes the gazer and the gazed establish a complicated power relationship. Frequently, women are in a position of being gazed at by others instead of taking the initiative to have a counter gaze. Consequently, they are confronted with abnormal self identity construction and alienation under the gazes of others and the self gaze brought by others’ gazes. This is fully revealed in the experiences of three representative female characters in Fitzgerald’s The Great Gatsby. This paper elaborates on their experiences and the relative consequences under gazes-deviation of femininity and other aspects, in the hope of providing a refreshing perspective for the study on this novel, gaze and gender issues.
文摘Many applications,including security systems,medical diagnostics,and human-computer interfaces,depend on eye gaze recognition.However,due to factors including individual variations,occlusions,and shifting illumination conditions,real-world scenarios continue to provide difficulties for accurate and consistent eye gaze recognition.This work is aimed at investigating the potential benefits of employing transfer learning to improve eye gaze detection ability and efficiency.Transfer learning is the process of fine-tuning pre-trained models on smaller,domain-specific datasets after they have been trained on larger datasets.We study several transfer learning algorithms and evaluate their effectiveness on eye gaze identification,including both Regression and Classification tasks,using a range of deep learning architectures,namely AlexNet,Visual Geometry Group(VGG),InceptionV3,and ResNet.In this study,we evaluate the effectiveness of transfer learning-basedmodels against models that were trained fromscratch using eye-gazing datasets on grounds of various performance and loss metrics such as Precision,Accuracy,and Mean Absolute Error.We investigate the effects of different pre-trainedmodels,dataset sizes,and domain gaps on the transfer learning process,and the findings of our study clarify the efficacy of transfer learning for eye gaze detection and offer suggestions for the most successful transfer learning strategies to apply in real-world situations.
基金supported by the National Natural Science Foundation of China(32471055 and 82171090)Shanghai Municipal Science and Technology Major Project(2018SHZDZX01)ZJLab,Shanghai Center for Brain Science and Brain-Inspired Technology,the Lingang Laboratory(LG-QS-202203-12).
文摘Throughout the lifespan,an animal can encounter predators frequently,thus the ability to avoid attacks from predators is crucial for its survival.The chances of evading danger can be greatly improved if the animal can respond immediately to the threat.Therefore,when an animal detects a threat through its visual system,it must quickly direct its gaze and attention toward the source of danger,assess the threat level,and take appropriate action.
文摘This paper probes into the transformation process and the deep meaning of the female characters in The Case of a Missing Seventeen and Gone Girl.The research shows that Wang Di and Amy realize the transformation from“being gazed at”to“gazer”through the disappearance in the Chinese and Western cultural backgrounds respectively.Wang Di’s secret resistance reflects the traditional culture’s suppression of women’s self-awareness,while Amy controls the development of the story through carefully planned strategies,revealing the complex position of women in the marriage system.These two works show the struggle and breakthrough of women in the gender power structure,and emphasize the necessity of gender equality consciousness.Through a detailed study of role analysis and narrative strategies,this paper provides a new perspective for understanding the survival and growth of modern women in gender relations.
文摘The biathlon,an Olympic sporting discipline that combines cross-country skiing with rifle marksmanship,entails considerable physiological demands,as well as fine motor control while shooting after intense exercise and under mental pressure.Although much of our knowledge about cross-country skiing is probably also applicable to the biathlon,carrying the rifle and shooting under stress make this discipline somewhat unique.The present review summarizes and examines the scientific literature related to biathlon performance,with a focus on physiological and biomechanical factors and shooting technique,as well as psychophysiological aspects of shooting performance.We conclude with suggestions for future research designed to extend our knowledge about the biathlon,which is presently quite limited.2018 Published by Elsevier B.V.on behalf of Shanghai University of Sport.This is an open access article under the CC BY-NC-ND license.(http://creativecommons.org/licenses/by-nc-nd/4.0/).
基金supported by National Natural Science Foundation of China(61533019,U1811463)Open Fund of the State Key Laboratory for Management and Control of Complex Systems,Institute of Automation,Chinese Academy of Sciences(Y6S9011F51)in part by the EPSRC Project(EP/N025849/1)
文摘Eye center localization is one of the most crucial and basic requirements for some human-computer interaction applications such as eye gaze estimation and eye tracking. There is a large body of works on this topic in recent years, but the accuracy still needs to be improved due to challenges in appearance such as the high variability of shapes, lighting conditions, viewing angles and possible occlusions. To address these problems and limitations, we propose a novel approach in this paper for the eye center localization with a fully convolutional network(FCN),which is an end-to-end and pixels-to-pixels network and can locate the eye center accurately. The key idea is to apply the FCN from the object semantic segmentation task to the eye center localization task since the problem of eye center localization can be regarded as a special semantic segmentation problem. We adapt contemporary FCN into a shallow structure with a large kernel convolutional block and transfer their performance from semantic segmentation to the eye center localization task by fine-tuning.Extensive experiments show that the proposed method outperforms the state-of-the-art methods in both accuracy and reliability of eye center localization. The proposed method has achieved a large performance improvement on the most challenging database and it thus provides a promising solution to some challenging applications.
文摘An active stereo vision system based on a model of neural pathways of human binocular motor system is proposed. With this model, it is guaranteed that the two cameras of the active stereo vision system can keep their lines of sight fixed on the same target object during smooth pursuit. This feature is very important for active stereo vision systems, since not only 3D reconstruction needs the two cameras have an overlapping field of vision, but also it can facilitate the 3D reconstruction algorithm. To evaluate the effectiveness of the proposed method, some software simulations are done to demonstrate the same target tracking characteristic in a virtual environment apt to mistracking easily. Here, mistracking means two eyes track two different objects separately. Then the proposed method is implemented in our active stereo vision system to perform real tracking task in a laboratory scene where several persons walk self-determining. Before the proposed model is implemented in the system, mistracking occurred frequently. After it is enabled, mistracking never occurred. The result shows that the vision system based on neural pathways of human binocular motor system can reliably avoid mistracking.
