The interferences,such as the background,eyebrows,eyelashes,eyeglass frames,illumination variations,and specular lens reflection pose challenges for pupil localization in natural scenes.In this paper,we propose a nove...The interferences,such as the background,eyebrows,eyelashes,eyeglass frames,illumination variations,and specular lens reflection pose challenges for pupil localization in natural scenes.In this paper,we propose a novel method comprising improved YOLOv8 and Illumination Adaptive Algorithm(IAA),for fast and accurate pupil localization in natural scenes.We introduced deformable convolution into the backbone of YOLOv8 to enable the model to extract the eye regions more accurately,thus avoiding the interference of background outside the eye on subsequent pupil localization.The IAA can reduce the interference of illumination variations and lens reflection by adjusting automatically the grayscale of the image according to the exposure.Experimental results verified that the improved YOLOv8 exhibited an eye detection accuracy(IOU≥0.5)of 90.2%,while the IAA leads to a 9.15%improvement on 5-pixels error ratio e5 with processing times in the tens of microseconds on GPU.Experimental results on the benchmark database CelebA show that the proposed method for pupil localization achieves an accuracy of 83.05%on e5 and achieves real-time performance of 210 FPS on GPU,outperforming other advanced methods.展开更多
The novel eye-based human-computer interaction(HCI) system aims to provide people, especially, disabled persons,a new way of communication with surroundings. It adopts a series of continual eye movements as input to p...The novel eye-based human-computer interaction(HCI) system aims to provide people, especially, disabled persons,a new way of communication with surroundings. It adopts a series of continual eye movements as input to perform simple control activities. Identification of eye movements is the crucial technology in these eye-based HCI systems. At present, researches on eye movement identification mainly focus on frontal face images. In fact, acquisition of non-frontal face images is more reasonable in real applications. In this paper, we discuss the identification process of eye movements from non-frontal face images. Firstly, the original head-shoulder images of 0?–±60?azimuths are sampled without any auxiliary light source. Secondly, the non-frontal face region is detected by using the Adaboost cascade classifiers. After that, we roughly extract eye windows by the integral projection function.Then, we propose a new method to calculate the x- y coordinates of the pupil center point by searching the minimal intensity value in the eye windows. According to the trajectory of the pupil center points, different eye movements(eye moving left, right, up or down)are successfully identified. A set of experiments is presented.展开更多
基金supported by Guangdong Yangfan Program for Innovative and Entrepreneurial Teams(Grant No.2017YT05G026)the Natural Science Foundation of China(Grant No.51975126)+1 种基金the Key Research and Development Program of Guangdong Province(Grant No.2019B090915001)the Natural Science Foundation of Guangdong Province(Grant No.2022A1515012605).
文摘The interferences,such as the background,eyebrows,eyelashes,eyeglass frames,illumination variations,and specular lens reflection pose challenges for pupil localization in natural scenes.In this paper,we propose a novel method comprising improved YOLOv8 and Illumination Adaptive Algorithm(IAA),for fast and accurate pupil localization in natural scenes.We introduced deformable convolution into the backbone of YOLOv8 to enable the model to extract the eye regions more accurately,thus avoiding the interference of background outside the eye on subsequent pupil localization.The IAA can reduce the interference of illumination variations and lens reflection by adjusting automatically the grayscale of the image according to the exposure.Experimental results verified that the improved YOLOv8 exhibited an eye detection accuracy(IOU≥0.5)of 90.2%,while the IAA leads to a 9.15%improvement on 5-pixels error ratio e5 with processing times in the tens of microseconds on GPU.Experimental results on the benchmark database CelebA show that the proposed method for pupil localization achieves an accuracy of 83.05%on e5 and achieves real-time performance of 210 FPS on GPU,outperforming other advanced methods.
基金supported by Innovation Program of Shanghai Municipal Education Commission of China(No.14YZ169)
文摘The novel eye-based human-computer interaction(HCI) system aims to provide people, especially, disabled persons,a new way of communication with surroundings. It adopts a series of continual eye movements as input to perform simple control activities. Identification of eye movements is the crucial technology in these eye-based HCI systems. At present, researches on eye movement identification mainly focus on frontal face images. In fact, acquisition of non-frontal face images is more reasonable in real applications. In this paper, we discuss the identification process of eye movements from non-frontal face images. Firstly, the original head-shoulder images of 0?–±60?azimuths are sampled without any auxiliary light source. Secondly, the non-frontal face region is detected by using the Adaboost cascade classifiers. After that, we roughly extract eye windows by the integral projection function.Then, we propose a new method to calculate the x- y coordinates of the pupil center point by searching the minimal intensity value in the eye windows. According to the trajectory of the pupil center points, different eye movements(eye moving left, right, up or down)are successfully identified. A set of experiments is presented.
