To track human across non-overlapping cameras in depression angles for applications such as multi-airplane visual human tracking and urban multi-camera surveillance,an adaptive human tracking method is proposed,focusi...To track human across non-overlapping cameras in depression angles for applications such as multi-airplane visual human tracking and urban multi-camera surveillance,an adaptive human tracking method is proposed,focusing on both feature representation and human tracking mechanism.Feature representation describes individual by using both improved local appearance descriptors and statistical geometric parameters.The improved feature descriptors can be extracted quickly and make the human feature more discriminative.Adaptive human tracking mechanism is based on feature representation and it arranges the human image blobs in field of view into matrix.Primary appearance models are created to include the maximum inter-camera appearance information captured from different visual angles.The persons appeared in camera are first filtered by statistical geometric parameters.Then the one among the filtered persons who has the maximum matching scale with the primary models is determined to be the target person.Subsequently,the image blobs of the target person are used to update and generate new primary appearance models for the next camera,thus being robust to visual angle changes.Experimental results prove the excellence of the feature representation and show the good generalization capability of tracking mechanism as well as its robustness to condition variables.展开更多
Of different model-based methods in vision based human tracking,many state of the art works focus on the stochastic optimization method to search in a very high dimensional space and try to find the optimal solution a...Of different model-based methods in vision based human tracking,many state of the art works focus on the stochastic optimization method to search in a very high dimensional space and try to find the optimal solution according to a proper likelihood function.Seldom works perform a framework of interactive multiple models (IMM) to track a human for challenging problems,such as uncertainty of motion styles,imprecise detection of feature points and ambiguity of joint location.This paper presents a two-layer filter framework based on IMM to track human motion.First,a method of model based points location is proposed to detect key feature points automatically and the filter in the first layer is performed to estimate the undetected points.Second,multiple models of motion are learned by the prior motion data with ridge regression and the IMM algorithm is used to estimate the quaternion vectors of joints rotation.Finally,experiments using real images sequences,simulation videos and 3D voxel data demonstrate that this human tracking framework is efficient.展开更多
With the high-speed development of transportation industry,highway traffic safety has become a considerable problem.Meanwhile,with the development of embedded system and hardware chip,in recent years,human eye detecti...With the high-speed development of transportation industry,highway traffic safety has become a considerable problem.Meanwhile,with the development of embedded system and hardware chip,in recent years,human eye detection eye tracking and positioning technology have been more and more widely used in man-machine interaction,security access control and visual detection.In this paper,the high parallelism of FPGA was utilized to realize an elliptical approximate real-time human eye tracking system,which was achieved by the series register structure and random sample consensus(RANSAC),thus improving the speed of image processing without using external memory.Because eye images acquired by the camera often generate a lot of noises due to uneven light and dark background,the preprocessing technologies such as color conversion,image filtering,histogram modification and image sharpening were adopted.In terms of feature extraction of images,the eye tracking algorithm in this paper adopted seven-section rectangular eye tracking characteristic method,which increased a section between the mouth and the nose on the basis of the traditional six-section method,so its recognition accuracy is much higher.It is convenient for the realization of hardware parallel system in FPGA.Finally,aiming at the accuracy and real-time performance of the design system,a more comprehensive simulation test was carried out.The human eye tracking system was verified on DE2-115 multimedia development platform,and the performance of VGA(resolution:640×480)images of 8-bit grayscale was tested.The results showed that the detection speed of this system was about 47 frames per second under the condition that the detection rate of human face(front face,no inclination)was 93%,which reached the real-time detection level.Additionally,the accuracy of eye tracking based on FPGA system was more than 95%,and it has achieved ideal results in real-time performance and robustness.展开更多
Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosen...Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosensory games, and human-computer interaction. However, vision-based pose tracking techniques usually raise privacy concerns, making human pose tracking without vision data usage an important problem. Thus, we propose using Radio Frequency Identification (RFID) as a pose tracking technique via a low-cost wearable sensing device. Although our prior work illustrated how deep learning could transfer RFID data into real-time human poses, generalization for different subjects remains challenging. This paper proposes a subject-adaptive technique to address this generalization problem. In the proposed system, termed Cycle-Pose, we leverage a cross-skeleton learning structure to improve the adaptability of the deep learning model to different human skeletons. Moreover, our novel cycle kinematic network is proposed for unpaired RFID and labeled pose data from different subjects. The Cycle-Pose system is implemented and evaluated by comparing its prototype with a traditional RFID pose tracking system. The experimental results demonstrate that Cycle-Pose can achieve lower estimation error and better subject generalization than the traditional system.展开更多
To cope with the problem of tracking a human head in a complicated scene,we propose a method that adopts human skin color and hair color integrated with a kind of particle filter named condensation algorithm.Firstly,a...To cope with the problem of tracking a human head in a complicated scene,we propose a method that adopts human skin color and hair color integrated with a kind of particle filter named condensation algorithm.Firstly,a novel method is presented to set up human head color model using skin color and hair color separately based on region growing.Compared with traditional human face model,this method is more precise and works well when human turns around and the face disappears in the image.Then a novel method is presented to use color model in condensation algorithm more effectively.In this method,a combination of edge detection result,color segmentation result and color edge detection result in an Omega window is used to measure the scale and position of human head in condensation.Experiments show that this approach can track human head in complicated scene even when human turns around or the distance of tracking a human head changes quickly.展开更多
Lots of progress has been made recently on 2 D human pose tracking with tracking-by-detection approaches. However,several challenges still remain in this area which is due to self-occlusions and the confusion between ...Lots of progress has been made recently on 2 D human pose tracking with tracking-by-detection approaches. However,several challenges still remain in this area which is due to self-occlusions and the confusion between the left and right limbs during tracking. In this work,a head orientation detection step is introduced into the tracking framework to serve as a complementary tool to assist human pose estimation. With the face orientation determined,the system can decide whether the left or right side of the human body is exactly visible and infer the state of the symmetric counterpart. By granting a higher priority for the completely visible side,the system can avoid double counting to a great extent when inferring body poses. The proposed framework is evaluated on the HumanEva dataset. The results show that it largely reduces the occurrence of double counting and distinguishes the left and right sides consistently.展开更多
To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For thi...To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For this pur-pose,human skeleton motion data are collected using a Ki-nect depth camera,and the obtained data are optimized via abnormal data elimination,smoothing,and normalization.MediaPipe extracts three-dimensional hand motion coordi-nates for accurate human posture tracking.Blender is used to build a virtual worker and site model,and the virtual worker motion is controlled based on the quaternion inverse kinematics algorithm while limiting the joint angle to en-hance the authenticity of motion simulation.Experimental results show that the system frame rate is stable at 60 frame/s,end-to-end delay is less than 20 ms,and virtual task comple-tion time is close to the real scene,verifying its engineering applicability.The proposed system can drive virtual work-ers to perform tasks and provide technical support for con-struction safety training.展开更多
Two-way feedback of human body was published in 1992. The sensation of two-way feedback of body is a special system of human reaction, which maintains and regulates symmetry and balance of human body. The human two-wa...Two-way feedback of human body was published in 1992. The sensation of two-way feedback of body is a special system of human reaction, which maintains and regulates symmetry and balance of human body. The human two-way feedback reacts to human health. For human overall health and delay decrepitude, it is necessary to pay attention to the stimulations (passive acceptance and initiative interventions) and relevant influences in human body and the stimulative effect. In this paper, the experimental research of stimulation and an example of two-way feedback in human body are given. And lay a foundation of prevention, medical treatment and hygiene of human overall health.展开更多
Surveillance systems can take various forms,but gait-based surveillance is emerging as a powerful approach due to its ability to identify individuals without requiring their cooperation.In the existing studies,several...Surveillance systems can take various forms,but gait-based surveillance is emerging as a powerful approach due to its ability to identify individuals without requiring their cooperation.In the existing studies,several approaches have been suggested for gait recognition;nevertheless,the performance of existing systems is often degraded in real-world conditions due to covariate factors such as occlusions,clothing changes,walking speed,and varying camera viewpoints.Furthermore,most existing research focuses on single-person gait recognition;however,counting,tracking,detecting,and recognizing individuals in dual-subject settings with occlusions remains a challenging task.Therefore,this research proposed a variant of an automated gait model for occluded dual-subject walk scenarios.More precisely,in the proposed method,we have designed a deep learning(DL)-based dual-subject gait model(DSG)involving three modules.The first module handles silhouette segmentation,localization,and counting(SLC)using Mask-RCNN with MobileNetV2.The next stage uses a Convolutional block attention module(CBAM)-based Siamese network for frame-level tracking with a modified gallery setting.Following the last,gait recognition based on regionbased deep learning is proposed for dual-subject gait recognition.The proposed method,tested on Shri Mata Vaishno Devi University(SMVDU)-Multi-Gait and Single-Gait datasets,shows strong performance with 94.00%segmentation,58.36%tracking,and 63.04%gait recognition accuracy in dual-subject walk scenarios.展开更多
基金funded by the Natural Science Foundation of Jiangsu Province(No.BK2012389)the National Natural Science Foundation of China(Nos.71303110,91024024)the Foundation of Graduate Innovation Center in NUAA(Nos.kfjj201471,kfjj201473)
文摘To track human across non-overlapping cameras in depression angles for applications such as multi-airplane visual human tracking and urban multi-camera surveillance,an adaptive human tracking method is proposed,focusing on both feature representation and human tracking mechanism.Feature representation describes individual by using both improved local appearance descriptors and statistical geometric parameters.The improved feature descriptors can be extracted quickly and make the human feature more discriminative.Adaptive human tracking mechanism is based on feature representation and it arranges the human image blobs in field of view into matrix.Primary appearance models are created to include the maximum inter-camera appearance information captured from different visual angles.The persons appeared in camera are first filtered by statistical geometric parameters.Then the one among the filtered persons who has the maximum matching scale with the primary models is determined to be the target person.Subsequently,the image blobs of the target person are used to update and generate new primary appearance models for the next camera,thus being robust to visual angle changes.Experimental results prove the excellence of the feature representation and show the good generalization capability of tracking mechanism as well as its robustness to condition variables.
基金the Research Fund for the Young Teacher of Shanghai(No.Z-2009-12)the New Teacher Fund of Shanghai University of Electric Power (No.K-2010-16)
文摘Of different model-based methods in vision based human tracking,many state of the art works focus on the stochastic optimization method to search in a very high dimensional space and try to find the optimal solution according to a proper likelihood function.Seldom works perform a framework of interactive multiple models (IMM) to track a human for challenging problems,such as uncertainty of motion styles,imprecise detection of feature points and ambiguity of joint location.This paper presents a two-layer filter framework based on IMM to track human motion.First,a method of model based points location is proposed to detect key feature points automatically and the filter in the first layer is performed to estimate the undetected points.Second,multiple models of motion are learned by the prior motion data with ridge regression and the IMM algorithm is used to estimate the quaternion vectors of joints rotation.Finally,experiments using real images sequences,simulation videos and 3D voxel data demonstrate that this human tracking framework is efficient.
文摘With the high-speed development of transportation industry,highway traffic safety has become a considerable problem.Meanwhile,with the development of embedded system and hardware chip,in recent years,human eye detection eye tracking and positioning technology have been more and more widely used in man-machine interaction,security access control and visual detection.In this paper,the high parallelism of FPGA was utilized to realize an elliptical approximate real-time human eye tracking system,which was achieved by the series register structure and random sample consensus(RANSAC),thus improving the speed of image processing without using external memory.Because eye images acquired by the camera often generate a lot of noises due to uneven light and dark background,the preprocessing technologies such as color conversion,image filtering,histogram modification and image sharpening were adopted.In terms of feature extraction of images,the eye tracking algorithm in this paper adopted seven-section rectangular eye tracking characteristic method,which increased a section between the mouth and the nose on the basis of the traditional six-section method,so its recognition accuracy is much higher.It is convenient for the realization of hardware parallel system in FPGA.Finally,aiming at the accuracy and real-time performance of the design system,a more comprehensive simulation test was carried out.The human eye tracking system was verified on DE2-115 multimedia development platform,and the performance of VGA(resolution:640×480)images of 8-bit grayscale was tested.The results showed that the detection speed of this system was about 47 frames per second under the condition that the detection rate of human face(front face,no inclination)was 93%,which reached the real-time detection level.Additionally,the accuracy of eye tracking based on FPGA system was more than 95%,and it has achieved ideal results in real-time performance and robustness.
基金supported in part by the US National Science Foundation(NSF)under Grants ECCS-1923163 and CNS-2107190through the Wireless Engineering Research and Education Center at Auburn University.
文摘Three-dimensional (3D) human pose tracking has recently attracted more and more attention in the computer vision field. Real-time pose tracking is highly useful in various domains such as video surveillance, somatosensory games, and human-computer interaction. However, vision-based pose tracking techniques usually raise privacy concerns, making human pose tracking without vision data usage an important problem. Thus, we propose using Radio Frequency Identification (RFID) as a pose tracking technique via a low-cost wearable sensing device. Although our prior work illustrated how deep learning could transfer RFID data into real-time human poses, generalization for different subjects remains challenging. This paper proposes a subject-adaptive technique to address this generalization problem. In the proposed system, termed Cycle-Pose, we leverage a cross-skeleton learning structure to improve the adaptability of the deep learning model to different human skeletons. Moreover, our novel cycle kinematic network is proposed for unpaired RFID and labeled pose data from different subjects. The Cycle-Pose system is implemented and evaluated by comparing its prototype with a traditional RFID pose tracking system. The experimental results demonstrate that Cycle-Pose can achieve lower estimation error and better subject generalization than the traditional system.
文摘To cope with the problem of tracking a human head in a complicated scene,we propose a method that adopts human skin color and hair color integrated with a kind of particle filter named condensation algorithm.Firstly,a novel method is presented to set up human head color model using skin color and hair color separately based on region growing.Compared with traditional human face model,this method is more precise and works well when human turns around and the face disappears in the image.Then a novel method is presented to use color model in condensation algorithm more effectively.In this method,a combination of edge detection result,color segmentation result and color edge detection result in an Omega window is used to measure the scale and position of human head in condensation.Experiments show that this approach can track human head in complicated scene even when human turns around or the distance of tracking a human head changes quickly.
文摘Lots of progress has been made recently on 2 D human pose tracking with tracking-by-detection approaches. However,several challenges still remain in this area which is due to self-occlusions and the confusion between the left and right limbs during tracking. In this work,a head orientation detection step is introduced into the tracking framework to serve as a complementary tool to assist human pose estimation. With the face orientation determined,the system can decide whether the left or right side of the human body is exactly visible and infer the state of the symmetric counterpart. By granting a higher priority for the completely visible side,the system can avoid double counting to a great extent when inferring body poses. The proposed framework is evaluated on the HumanEva dataset. The results show that it largely reduces the occurrence of double counting and distinguishes the left and right sides consistently.
基金The Eighth National “Ten Thousand Talents Plan for Top Young Talents” of Chinathe National Natural Science Foundation of China (No. 52478117, 52378120)。
文摘To improve the safety of construction workers and help workers remotely control humanoid robots in construc-tion,this study designs and implements a computer vision based virtual construction simulation system.For this pur-pose,human skeleton motion data are collected using a Ki-nect depth camera,and the obtained data are optimized via abnormal data elimination,smoothing,and normalization.MediaPipe extracts three-dimensional hand motion coordi-nates for accurate human posture tracking.Blender is used to build a virtual worker and site model,and the virtual worker motion is controlled based on the quaternion inverse kinematics algorithm while limiting the joint angle to en-hance the authenticity of motion simulation.Experimental results show that the system frame rate is stable at 60 frame/s,end-to-end delay is less than 20 ms,and virtual task comple-tion time is close to the real scene,verifying its engineering applicability.The proposed system can drive virtual work-ers to perform tasks and provide technical support for con-struction safety training.
文摘Two-way feedback of human body was published in 1992. The sensation of two-way feedback of body is a special system of human reaction, which maintains and regulates symmetry and balance of human body. The human two-way feedback reacts to human health. For human overall health and delay decrepitude, it is necessary to pay attention to the stimulations (passive acceptance and initiative interventions) and relevant influences in human body and the stimulative effect. In this paper, the experimental research of stimulation and an example of two-way feedback in human body are given. And lay a foundation of prevention, medical treatment and hygiene of human overall health.
基金supported by the MSIT(Ministry of Science and ICT),Republic of Korea,under the Convergence Security Core Talent Training Business Support Program(IITP-2025-RS-2023-00266605)supervised by the IITP(Institute for Information&Communications Technology Planning&Evaluation).
文摘Surveillance systems can take various forms,but gait-based surveillance is emerging as a powerful approach due to its ability to identify individuals without requiring their cooperation.In the existing studies,several approaches have been suggested for gait recognition;nevertheless,the performance of existing systems is often degraded in real-world conditions due to covariate factors such as occlusions,clothing changes,walking speed,and varying camera viewpoints.Furthermore,most existing research focuses on single-person gait recognition;however,counting,tracking,detecting,and recognizing individuals in dual-subject settings with occlusions remains a challenging task.Therefore,this research proposed a variant of an automated gait model for occluded dual-subject walk scenarios.More precisely,in the proposed method,we have designed a deep learning(DL)-based dual-subject gait model(DSG)involving three modules.The first module handles silhouette segmentation,localization,and counting(SLC)using Mask-RCNN with MobileNetV2.The next stage uses a Convolutional block attention module(CBAM)-based Siamese network for frame-level tracking with a modified gallery setting.Following the last,gait recognition based on regionbased deep learning is proposed for dual-subject gait recognition.The proposed method,tested on Shri Mata Vaishno Devi University(SMVDU)-Multi-Gait and Single-Gait datasets,shows strong performance with 94.00%segmentation,58.36%tracking,and 63.04%gait recognition accuracy in dual-subject walk scenarios.
