Because of its characteristics of simple algorithm and hardware, optical flow-based motion estimation has become a hot research field, especially in GPS-denied environment. Optical flow could be used to obtain the air...Because of its characteristics of simple algorithm and hardware, optical flow-based motion estimation has become a hot research field, especially in GPS-denied environment. Optical flow could be used to obtain the aircraft motion information, but the six-(degree of freedom)(6-DOF) motion still couldn't be accurately estimated by existing methods. The purpose of this work is to provide a motion estimation method based on optical flow from forward and down looking cameras, which doesn't rely on the assumption of level flight. First, the distribution and decoupling method of optical flow from forward camera are utilized to get attitude. Then, the resulted angular velocities are utilized to obtain the translational optical flow of the down camera, which can eliminate the influence of rotational motion on velocity estimation. Besides, the translational motion estimation equation is simplified by establishing the relation between the depths of feature points and the aircraft altitude. Finally, simulation results show that the method presented is accurate and robust.展开更多
The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Ki...The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.展开更多
To enhance the image motion compensation accuracy of off-axis three-mirror anastigmatic( TMA)three-line array aerospace mapping cameras,a new method of image motion velocity field modeling is proposed in this paper. F...To enhance the image motion compensation accuracy of off-axis three-mirror anastigmatic( TMA)three-line array aerospace mapping cameras,a new method of image motion velocity field modeling is proposed in this paper. Firstly,based on the imaging principle of mapping cameras,an analytical expression of image motion velocity of off-axis TMA three-line array aerospace mapping cameras is deduced from different coordinate systems we established and the attitude dynamics principle. Then,the case of a three-line array mapping camera is studied,in which the simulation of the focal plane image motion velocity fields of the forward-view camera,the nadir-view camera and the backward-view camera are carried out,and the optimization schemes for image motion velocity matching and drift angle matching are formulated according the simulation results. Finally,this method is verified with a dynamic imaging experimental system. The results are indicative of that when image motion compensation for nadir-view camera is conducted using the proposed image motion velocity field model,the line pair of target images at Nyquist frequency is clear and distinguishable. Under the constraint that modulation transfer function( MTF) reduces by 5%,when the horizontal frequencies of the forward-view camera and the backward-view camera are adjusted uniformly according to the proposed image motion velocity matching scheme,the time delay integration( TDI) stages reach 6 at most. When the TDI stages are more than 6,the three groups of camera will independently undergo horizontal frequency adjustment. However, when the proposed drift angle matching scheme is adopted for uniform drift angle adjustment,the number of TDI stages will not exceed 81. The experimental results have demonstrated the validity and accuracy of the proposed image motion velocity field model and matching optimization scheme,providing reliable basis for on-orbit image motion compensation of aerospace mapping cameras.展开更多
Fatigue cracks that develop in civil infrastructure such as steel bridges due to repetitive loads pose a major threat to structural integrity.Despite being the most common practice for fatigue crack detection,human vi...Fatigue cracks that develop in civil infrastructure such as steel bridges due to repetitive loads pose a major threat to structural integrity.Despite being the most common practice for fatigue crack detection,human visual inspection is known to be labor intensive,time-consuming,and prone to error.In this study,a computer vision-based fatigue crack detection approach using a short video recorded under live loads by a moving consumer-grade camera is presented.The method detects fatigue crack by tracking surface motion and identifies the differential motion pattern caused by opening and closing of the fatigue crack.However,the global motion introduced by a moving camera in the recorded video is typically far greater than the actual motion associated with fatigue crack opening/closing,leading to false detection results.To overcome the challenge,global motion compensation(GMC)techniques are introduced to compensate for camera-induced movement.In particular,hierarchical model-based motion estimation is adopted for 2D videos with simple geometry and a new method is developed by extending the bundled camera paths approach for 3D videos with complex geometry.The proposed methodology is validated using two laboratory test setups for both in-plane and out-of-plane fatigue cracks.The results confirm the importance of motion compensation for both 2D and 3D videos and demonstrate the effectiveness of the proposed GMC methods as well as the subsequent crack detection algorithm.展开更多
Imagine that hundreds of video streams, taken by mobile phones during a rock concert, are uploaded to a server. One attractive application of such prominent dataset is to allow a user to create his own video with a de...Imagine that hundreds of video streams, taken by mobile phones during a rock concert, are uploaded to a server. One attractive application of such prominent dataset is to allow a user to create his own video with a deliberately chosen but virtual camera trajectory. In this paper we present algorithms for the main sub-tasks (spatial calibration, image interpolation) related to this problem. Calibration: Spatial calibration of individual video streams is one of the most basic tasks related to creating such a video. At its core, this requires to estimate the pairwise relative geometry of images taken by different cameras. It is also known as the relative pose problem [1], and is fundamental to many computer vision algorithms. In practice, efficiency and robustness are of highest relevance for big data applications such as the ones addressed in the EU-FET_SME project SceneNet. In this paper, we present an improved algorithm that exploits additional data from inertial sensors, such as accelerometer, magnetometer or gyroscopes, which by now are available in most mobile phones. Experimental results on synthetic and real data demonstrate the accuracy and efficiency of our algorithm. Interpolation: Given the calibrated cameras, we present a second algorithm that generates novel synthetic images along a predefined specific camera trajectory. Each frame is produced from two “neighboring” video streams that are selected from the data base. The interpolation algorithm is then based on the point cloud reconstructed in the spatial calibration phase and iteratively projects triangular patches from the existing images into the new view. We present convincing images synthesized with the proposed algorithm.展开更多
In the cotton factories ginning process bales of raw cotton in cotton tube transporting through the air has written in the article. As a result, cotton with cotton in the air separated by a separator device with air t...In the cotton factories ginning process bales of raw cotton in cotton tube transporting through the air has written in the article. As a result, cotton with cotton in the air separated by a separator device with air traffic, has an important theoretical study. First of all, the air flow that affects the dynamic pressure and laws is based on the model established in this study and the results are obtained.展开更多
Objective： To observe the tested results of the segmental range of motion （ROM） of lumbar spine by charge couple device （CCD）-based system for 3-dimensional real-time positioning （CCD system）, and to analyze it...Objective： To observe the tested results of the segmental range of motion （ROM） of lumbar spine by charge couple device （CCD）-based system for 3-dimensional real-time positioning （CCD system）, and to analyze its clinical significance. Methods： Seven patients with lumbar joint dysfunction and 8 healthy subjects were tested twice by the CCD-based system with an interval of 10 min. Results： The ROM of the patients was obviously lesser than that of the healthy subjects. The measuring data of segmental ROM of lumbar spine by CCD system is correlated significantly to the same data checked later on the same subjects in every direction of the movements. The differences between two checks are usually less than 1 degree. Conclusion： Specially designed CCD based system for 3-dimensional real-time positioning could objectively reflect the segmental ROM of lumbar spine. The system would be of great clinical significance in the assessment of the biomechanical dysfunction of lumbar spine and the effect of the treatment applied.展开更多
In this paper,an innovative 3D motion parameters estimation method from stereo image sequences based on infrared(IR) reflective markers is presented.It was assumed that two high speed CCD cameras had been calibrated p...In this paper,an innovative 3D motion parameters estimation method from stereo image sequences based on infrared(IR) reflective markers is presented.It was assumed that two high speed CCD cameras had been calibrated previously.The method consists of the following steps:1) the coordinate of several markers and depth map for each stereo pair was determined from the sequences of stereo images by relations of markers' coordinate the correspondence between markers was established,2) the 3D motion parameters of the target was computed based upon the matched markers' coordinate,and 3) translated 3D motion parameters estimation into the problem of least square according to the movement model of the object to be measured.Without using line,curve or corner correspondence,this method can calculate the depth of these markers feature easily and quickly in contrast to traditional approaches.The two CCD cameras work on 200 f/s,and each processing cost time is about 3 ms.It was found that,by using several markers and a large number of stereo images,this method can improve the computational speed,robustness and numerical accuracy of the motion parameters in comparison with traditional methods.The virtual simulation experiment was conducted using synthesized stereo image sequences based on 6-DOF motion platform and the experimental results proved the validity of our approach and showed that the translation and rotation precision is up to 0.1 mm and 0.1°.展开更多
针对在无人机高速运动场景下,相关滤波跟踪算法存在边界效应和滤波器退化等问题,提出了一种融合相机运动补偿的自适应时空正则化的无人机跟踪相关滤波算法。首先,在相关滤波器中设计了一种自适应空间正则化方法来缓解边界效应;其次,为...针对在无人机高速运动场景下,相关滤波跟踪算法存在边界效应和滤波器退化等问题,提出了一种融合相机运动补偿的自适应时空正则化的无人机跟踪相关滤波算法。首先,在相关滤波器中设计了一种自适应空间正则化方法来缓解边界效应;其次,为防止滤波器退化,设计了基于高置信度样本的时间正则化方法及优化更新策略,同时利用交替方向乘子法(Alternating Direction Method of Multipliers,ADMM)优化目标函数的求解,保证了算法的运行效率;然后,设计了快速尺度滤波器,实现目标尺度的快速估计;最后,提出了一种鲁棒性的跟踪不确定性规则及基于相机运动补偿的重检测器,使得目标丢失后可及时恢复。实验结果表明,提出的算法在无人机数据集上的精确度和成功率分别可达0.739和0.634,与ECO_HC(Efficient Convolution Operators-Hand Crafted)算法相比分别提升了1.65%和6.9%,与同类主流算法相比,有效地提升了跟踪时的精确度和成功率,经历快速运动、目标遮挡等情况时应用该算法的无人机也表现出良好的跟踪性能。展开更多
基金Project(2012CB720003)supported by the National Basic Research Program of ChinaProjects(61320106010,61127007,61121003,61573019)supported by the National Natural Science Foundation of ChinaProject(2013DFE13040)supported by the Special Program for International Science and Technology Cooperation from Ministry of Science and Technology of China
文摘Because of its characteristics of simple algorithm and hardware, optical flow-based motion estimation has become a hot research field, especially in GPS-denied environment. Optical flow could be used to obtain the aircraft motion information, but the six-(degree of freedom)(6-DOF) motion still couldn't be accurately estimated by existing methods. The purpose of this work is to provide a motion estimation method based on optical flow from forward and down looking cameras, which doesn't rely on the assumption of level flight. First, the distribution and decoupling method of optical flow from forward camera are utilized to get attitude. Then, the resulted angular velocities are utilized to obtain the translational optical flow of the down camera, which can eliminate the influence of rotational motion on velocity estimation. Besides, the translational motion estimation equation is simplified by establishing the relation between the depths of feature points and the aircraft altitude. Finally, simulation results show that the method presented is accurate and robust.
文摘The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.
基金Sponsored by the National High Technology Research and Development Program of China(Grant No.863-2-5-1-13B)the Jilin Province Science and Technology Development Plan Item(Grant No.20130522107JH)
文摘To enhance the image motion compensation accuracy of off-axis three-mirror anastigmatic( TMA)three-line array aerospace mapping cameras,a new method of image motion velocity field modeling is proposed in this paper. Firstly,based on the imaging principle of mapping cameras,an analytical expression of image motion velocity of off-axis TMA three-line array aerospace mapping cameras is deduced from different coordinate systems we established and the attitude dynamics principle. Then,the case of a three-line array mapping camera is studied,in which the simulation of the focal plane image motion velocity fields of the forward-view camera,the nadir-view camera and the backward-view camera are carried out,and the optimization schemes for image motion velocity matching and drift angle matching are formulated according the simulation results. Finally,this method is verified with a dynamic imaging experimental system. The results are indicative of that when image motion compensation for nadir-view camera is conducted using the proposed image motion velocity field model,the line pair of target images at Nyquist frequency is clear and distinguishable. Under the constraint that modulation transfer function( MTF) reduces by 5%,when the horizontal frequencies of the forward-view camera and the backward-view camera are adjusted uniformly according to the proposed image motion velocity matching scheme,the time delay integration( TDI) stages reach 6 at most. When the TDI stages are more than 6,the three groups of camera will independently undergo horizontal frequency adjustment. However, when the proposed drift angle matching scheme is adopted for uniform drift angle adjustment,the number of TDI stages will not exceed 81. The experimental results have demonstrated the validity and accuracy of the proposed image motion velocity field model and matching optimization scheme,providing reliable basis for on-orbit image motion compensation of aerospace mapping cameras.
基金NCHRP Project,IDEA 223:Fatigue Crack Inspection using Computer Vision and Augmented Reality。
文摘Fatigue cracks that develop in civil infrastructure such as steel bridges due to repetitive loads pose a major threat to structural integrity.Despite being the most common practice for fatigue crack detection,human visual inspection is known to be labor intensive,time-consuming,and prone to error.In this study,a computer vision-based fatigue crack detection approach using a short video recorded under live loads by a moving consumer-grade camera is presented.The method detects fatigue crack by tracking surface motion and identifies the differential motion pattern caused by opening and closing of the fatigue crack.However,the global motion introduced by a moving camera in the recorded video is typically far greater than the actual motion associated with fatigue crack opening/closing,leading to false detection results.To overcome the challenge,global motion compensation(GMC)techniques are introduced to compensate for camera-induced movement.In particular,hierarchical model-based motion estimation is adopted for 2D videos with simple geometry and a new method is developed by extending the bundled camera paths approach for 3D videos with complex geometry.The proposed methodology is validated using two laboratory test setups for both in-plane and out-of-plane fatigue cracks.The results confirm the importance of motion compensation for both 2D and 3D videos and demonstrate the effectiveness of the proposed GMC methods as well as the subsequent crack detection algorithm.
文摘Imagine that hundreds of video streams, taken by mobile phones during a rock concert, are uploaded to a server. One attractive application of such prominent dataset is to allow a user to create his own video with a deliberately chosen but virtual camera trajectory. In this paper we present algorithms for the main sub-tasks (spatial calibration, image interpolation) related to this problem. Calibration: Spatial calibration of individual video streams is one of the most basic tasks related to creating such a video. At its core, this requires to estimate the pairwise relative geometry of images taken by different cameras. It is also known as the relative pose problem [1], and is fundamental to many computer vision algorithms. In practice, efficiency and robustness are of highest relevance for big data applications such as the ones addressed in the EU-FET_SME project SceneNet. In this paper, we present an improved algorithm that exploits additional data from inertial sensors, such as accelerometer, magnetometer or gyroscopes, which by now are available in most mobile phones. Experimental results on synthetic and real data demonstrate the accuracy and efficiency of our algorithm. Interpolation: Given the calibrated cameras, we present a second algorithm that generates novel synthetic images along a predefined specific camera trajectory. Each frame is produced from two “neighboring” video streams that are selected from the data base. The interpolation algorithm is then based on the point cloud reconstructed in the spatial calibration phase and iteratively projects triangular patches from the existing images into the new view. We present convincing images synthesized with the proposed algorithm.
文摘In the cotton factories ginning process bales of raw cotton in cotton tube transporting through the air has written in the article. As a result, cotton with cotton in the air separated by a separator device with air traffic, has an important theoretical study. First of all, the air flow that affects the dynamic pressure and laws is based on the model established in this study and the results are obtained.
文摘Objective： To observe the tested results of the segmental range of motion （ROM） of lumbar spine by charge couple device （CCD）-based system for 3-dimensional real-time positioning （CCD system）, and to analyze its clinical significance. Methods： Seven patients with lumbar joint dysfunction and 8 healthy subjects were tested twice by the CCD-based system with an interval of 10 min. Results： The ROM of the patients was obviously lesser than that of the healthy subjects. The measuring data of segmental ROM of lumbar spine by CCD system is correlated significantly to the same data checked later on the same subjects in every direction of the movements. The differences between two checks are usually less than 1 degree. Conclusion： Specially designed CCD based system for 3-dimensional real-time positioning could objectively reflect the segmental ROM of lumbar spine. The system would be of great clinical significance in the assessment of the biomechanical dysfunction of lumbar spine and the effect of the treatment applied.
基金Sponsored by the National Natural Science Foundation of China(Grant No. 50275040)
文摘In this paper,an innovative 3D motion parameters estimation method from stereo image sequences based on infrared(IR) reflective markers is presented.It was assumed that two high speed CCD cameras had been calibrated previously.The method consists of the following steps:1) the coordinate of several markers and depth map for each stereo pair was determined from the sequences of stereo images by relations of markers' coordinate the correspondence between markers was established,2) the 3D motion parameters of the target was computed based upon the matched markers' coordinate,and 3) translated 3D motion parameters estimation into the problem of least square according to the movement model of the object to be measured.Without using line,curve or corner correspondence,this method can calculate the depth of these markers feature easily and quickly in contrast to traditional approaches.The two CCD cameras work on 200 f/s,and each processing cost time is about 3 ms.It was found that,by using several markers and a large number of stereo images,this method can improve the computational speed,robustness and numerical accuracy of the motion parameters in comparison with traditional methods.The virtual simulation experiment was conducted using synthesized stereo image sequences based on 6-DOF motion platform and the experimental results proved the validity of our approach and showed that the translation and rotation precision is up to 0.1 mm and 0.1°.
文摘针对在无人机高速运动场景下,相关滤波跟踪算法存在边界效应和滤波器退化等问题,提出了一种融合相机运动补偿的自适应时空正则化的无人机跟踪相关滤波算法。首先,在相关滤波器中设计了一种自适应空间正则化方法来缓解边界效应;其次,为防止滤波器退化,设计了基于高置信度样本的时间正则化方法及优化更新策略,同时利用交替方向乘子法(Alternating Direction Method of Multipliers,ADMM)优化目标函数的求解,保证了算法的运行效率;然后,设计了快速尺度滤波器,实现目标尺度的快速估计;最后,提出了一种鲁棒性的跟踪不确定性规则及基于相机运动补偿的重检测器,使得目标丢失后可及时恢复。实验结果表明,提出的算法在无人机数据集上的精确度和成功率分别可达0.739和0.634,与ECO_HC(Efficient Convolution Operators-Hand Crafted)算法相比分别提升了1.65%和6.9%,与同类主流算法相比,有效地提升了跟踪时的精确度和成功率,经历快速运动、目标遮挡等情况时应用该算法的无人机也表现出良好的跟踪性能。
