Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting w...Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera.展开更多
The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Ki...The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.展开更多
With many advantages such as non-invasive,safe and quick effect,focused ultrasound lipolysis stands out among many fat-removing methods.However,during the whole process,the doctor needs to hold the ultrasound transduc...With many advantages such as non-invasive,safe and quick effect,focused ultrasound lipolysis stands out among many fat-removing methods.However,during the whole process,the doctor needs to hold the ultrasound transducer and press it on the patient’s skin with a large pressure for a long time;thus the probability of muscle and bone damage for doctors is greatly increased.To reduce the occurrence of doctors’occupational diseases,a depth camera-based ultrasonic lipolysis robot system is proposed to realize robot-assisted automatic ultrasonic lipolysis operation.The system is composed of RealSense depth camera,KUKA LBR Med seven-axis robotic arm,PC host,and ultrasonic lipolysis instrument.The whole operation includes two parts:preoperative planning and intraoperative operation.In preoperative planning,the treatment area is selected in the camera image by the doctor;then the system automatically plans uniformly distributed treatment points in the treatment area.At the same time,the skin normal vector is calculated to determine the end posture of the robot,so that the ultrasound transducer can be pressed down in the normal direction of skin.During the intraoperative operation,the robot is controlled to arrive at the treatment point in turn.Meanwhile,the patient’s movement can be detected by the depth camera,and the path of robot is adjusted in real time so that the robot can track the movement of patient,thereby ensuring the accuracy of the ultrasonic lipolysis operation.Finally,the human body model experiment is conducted.The results show that the maximum error of the robot operation is within 5mm,average error is 3.1mm,and the treatment points of the robot operation are more uniform than those of manual operation.Therefore,the system can replace the doctor and achieve autonomous ultrasonic lipolysis to reduce the doctor’s labor intensity.展开更多
This paper proposes a depth measurement error model for consumer depth cameras such as the Microsoft Kinect, and a corresponding calibration method. These devices were originally designed as video game interfaces, and...This paper proposes a depth measurement error model for consumer depth cameras such as the Microsoft Kinect, and a corresponding calibration method. These devices were originally designed as video game interfaces, and their output depth maps usually lack sufficient accuracy for 3 D measurement.Models have been proposed to reduce these depth errors, but they only consider camera-related causes.Since the depth sensors are based on projectorcamera systems, we should also consider projectorrelated causes. Also, previous models require disparity observations, which are usually not output by such sensors, so cannot be employed in practice. We give an alternative error model for projector-camera based consumer depth cameras, based on their depth measurement algorithm, and intrinsic parameters of the camera and the projector; it does not need disparity values. We also give a corresponding new parameter estimation method which simply needs observation of a planar board. Our calibrated error model allows use of a consumer depth sensor as a 3 D measuring device.Experimental results show the validity and effectiveness of the error model and calibration procedure.展开更多
Fast assessment of the initial carbon to nitrogen ratio(C/N)of organic fraction of municipal solid waste(OFMSW)is an important prerequisite for automatic composting control to improve efficiency and stability of the b...Fast assessment of the initial carbon to nitrogen ratio(C/N)of organic fraction of municipal solid waste(OFMSW)is an important prerequisite for automatic composting control to improve efficiency and stability of the bioconversion process.In this study,a novel approach was proposed to estimate the C/N of OFMSW,where an instance segmentation model was applied to predict the masks for the waste images.Then,by combining the instance segmentation model with the depth-camera-based volume calculation algorithm,the volumes occupied by each type of waste were obtained,therefore the C/N could be estimated based on the properties of each type of waste.First,an instance segmentation dataset including three common classes of OFMSW was built to train mask region-based convolutional neural networks(Mask R-CNN)model.Second,a volume measurement algorithm was proposed,where the measurement result of the object was derived by accumulating the volumes of small rectangular cuboids whose bottom area was calculated with the projection property.Then the calculated volume was corrected with linear regression models.The results showed that the trained instance segmentation model performed well with average precision scores AP_(50)=82.9,AP_(75)=72.5,and mask intersection over unit(Mask IoU)=45.1.A high correlation was found between the estimated C/N and the ground truth with a coefficient of determination R2=0.97 and root mean square error RMSE=0.10.The relative average error was 0.42%and the maximum error was only 1.71%,which indicated this approach has potential for practical applications.展开更多
Purpose:To evaluate the relationship between the position of the focal adjustment knob of a fundus camera and refractive error and biometric data as measured in the same eye.Methods:Normal eyes of patients presenting ...Purpose:To evaluate the relationship between the position of the focal adjustment knob of a fundus camera and refractive error and biometric data as measured in the same eye.Methods:Normal eyes of patients presenting to clinics at the Beijing Tongren Hospital were examined with a non-mydriatic fundus camera.The position on the focal scale of a knob adjusting the distance between the camera lens and film plane,used to adjust focus the image of the patients fundus relative to the refractive power of the eye,was recorded in degrees.Ocular biometry and refractometry were performed on the same eyes.Results:The study included 136 subjects with a mean age of 36.5 ±19.6 years and a mean refractive error of-1.31 ±2.77 diopters.In univariate analysis,the position of the adjustment knob was significantly associated with refractive error.(P < 0.001;correlation coefficient r=-0.77),axial length.(P<0.001;r=0.65) and anterior chamber depth (P<0.001;r=0.48).After adjustment for age,anterior chamber depth decreased by 0.01 mm(95% confidence interval:0.003,0.017) for change per degree in the position of the adjustment knob.Conclusion:A fundus camera can be used to estimate anterior chamber depth,axial length and refractive error.In a screening setting,a fundus camera operated by a technician may be helpful to detect a shallow anterior chamber and evaluate a potential risk factor for primary angle closure.展开更多
High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to...High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to obtain an image including the target.Then,two different images with two different view points are obtained,and they are used in calculating the position deviation of the image's pixels based on triangular measurement.Finally,the three-dimensional coordinate of the object is reconstructed.All the video data is processed by using field-programmable gate array(FPGA)in real-time.Hardware implementation speeds up the performance and reduces the power,thus,this hardware architecture can be applied in the portable environment.展开更多
基金National Natural Science Foundation of China(61732016).
文摘Three-dimensional(3D)modeling is an important topic in computer graphics and computer vision.In recent years,the introduction of consumer-grade depth cameras has resulted in profound advances in 3D modeling.Starting with the basic data structure,this survey reviews the latest developments of 3D modeling based on depth cameras,including research works on camera tracking,3D object and scene reconstruction,and high-quality texture reconstruction.We also discuss the future work and possible solutions for 3D modeling based on the depth camera.
文摘The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.
基金the National Natural Science Foundation of China(Nos.61973211,51911540479 and M-0221)the Research Project of Institute of Medical Robotics of Shanghai Jiao Tong Universitythe Project of Science and Technology Commission of Shanghai Municipality(No.20DZ2220400)。
文摘With many advantages such as non-invasive,safe and quick effect,focused ultrasound lipolysis stands out among many fat-removing methods.However,during the whole process,the doctor needs to hold the ultrasound transducer and press it on the patient’s skin with a large pressure for a long time;thus the probability of muscle and bone damage for doctors is greatly increased.To reduce the occurrence of doctors’occupational diseases,a depth camera-based ultrasonic lipolysis robot system is proposed to realize robot-assisted automatic ultrasonic lipolysis operation.The system is composed of RealSense depth camera,KUKA LBR Med seven-axis robotic arm,PC host,and ultrasonic lipolysis instrument.The whole operation includes two parts:preoperative planning and intraoperative operation.In preoperative planning,the treatment area is selected in the camera image by the doctor;then the system automatically plans uniformly distributed treatment points in the treatment area.At the same time,the skin normal vector is calculated to determine the end posture of the robot,so that the ultrasound transducer can be pressed down in the normal direction of skin.During the intraoperative operation,the robot is controlled to arrive at the treatment point in turn.Meanwhile,the patient’s movement can be detected by the depth camera,and the path of robot is adjusted in real time so that the robot can track the movement of patient,thereby ensuring the accuracy of the ultrasonic lipolysis operation.Finally,the human body model experiment is conducted.The results show that the maximum error of the robot operation is within 5mm,average error is 3.1mm,and the treatment points of the robot operation are more uniform than those of manual operation.Therefore,the system can replace the doctor and achieve autonomous ultrasonic lipolysis to reduce the doctor’s labor intensity.
基金supported by the JST CREST“Behavior Understanding based on Intention-Gait Model”project
文摘This paper proposes a depth measurement error model for consumer depth cameras such as the Microsoft Kinect, and a corresponding calibration method. These devices were originally designed as video game interfaces, and their output depth maps usually lack sufficient accuracy for 3 D measurement.Models have been proposed to reduce these depth errors, but they only consider camera-related causes.Since the depth sensors are based on projectorcamera systems, we should also consider projectorrelated causes. Also, previous models require disparity observations, which are usually not output by such sensors, so cannot be employed in practice. We give an alternative error model for projector-camera based consumer depth cameras, based on their depth measurement algorithm, and intrinsic parameters of the camera and the projector; it does not need disparity values. We also give a corresponding new parameter estimation method which simply needs observation of a planar board. Our calibrated error model allows use of a consumer depth sensor as a 3 D measuring device.Experimental results show the validity and effectiveness of the error model and calibration procedure.
基金funded by the National Key Research and Development Program of China(Grant No.2018YFD0200800)Key Research and Development Program of Hunan Province(Grant No.2018GK2013)+1 种基金Hunan Modern Agricultural Industry Technology Program(Grant No.201926)Innovation and Entrepreneurship Training Program of Hunan Agricultural University(Grant No.2019062x).
文摘Fast assessment of the initial carbon to nitrogen ratio(C/N)of organic fraction of municipal solid waste(OFMSW)is an important prerequisite for automatic composting control to improve efficiency and stability of the bioconversion process.In this study,a novel approach was proposed to estimate the C/N of OFMSW,where an instance segmentation model was applied to predict the masks for the waste images.Then,by combining the instance segmentation model with the depth-camera-based volume calculation algorithm,the volumes occupied by each type of waste were obtained,therefore the C/N could be estimated based on the properties of each type of waste.First,an instance segmentation dataset including three common classes of OFMSW was built to train mask region-based convolutional neural networks(Mask R-CNN)model.Second,a volume measurement algorithm was proposed,where the measurement result of the object was derived by accumulating the volumes of small rectangular cuboids whose bottom area was calculated with the projection property.Then the calculated volume was corrected with linear regression models.The results showed that the trained instance segmentation model performed well with average precision scores AP_(50)=82.9,AP_(75)=72.5,and mask intersection over unit(Mask IoU)=45.1.A high correlation was found between the estimated C/N and the ground truth with a coefficient of determination R2=0.97 and root mean square error RMSE=0.10.The relative average error was 0.42%and the maximum error was only 1.71%,which indicated this approach has potential for practical applications.
文摘Purpose:To evaluate the relationship between the position of the focal adjustment knob of a fundus camera and refractive error and biometric data as measured in the same eye.Methods:Normal eyes of patients presenting to clinics at the Beijing Tongren Hospital were examined with a non-mydriatic fundus camera.The position on the focal scale of a knob adjusting the distance between the camera lens and film plane,used to adjust focus the image of the patients fundus relative to the refractive power of the eye,was recorded in degrees.Ocular biometry and refractometry were performed on the same eyes.Results:The study included 136 subjects with a mean age of 36.5 ±19.6 years and a mean refractive error of-1.31 ±2.77 diopters.In univariate analysis,the position of the adjustment knob was significantly associated with refractive error.(P < 0.001;correlation coefficient r=-0.77),axial length.(P<0.001;r=0.65) and anterior chamber depth (P<0.001;r=0.48).After adjustment for age,anterior chamber depth decreased by 0.01 mm(95% confidence interval:0.003,0.017) for change per degree in the position of the adjustment knob.Conclusion:A fundus camera can be used to estimate anterior chamber depth,axial length and refractive error.In a screening setting,a fundus camera operated by a technician may be helpful to detect a shallow anterior chamber and evaluate a potential risk factor for primary angle closure.
文摘High performance hardware architecture for depth measurement by using binocular-camera is proposed.In the system,at first,video streams of the target are captured by left and right charge-coupled device(CCD)cameras to obtain an image including the target.Then,two different images with two different view points are obtained,and they are used in calculating the position deviation of the image's pixels based on triangular measurement.Finally,the three-dimensional coordinate of the object is reconstructed.All the video data is processed by using field-programmable gate array(FPGA)in real-time.Hardware implementation speeds up the performance and reduces the power,thus,this hardware architecture can be applied in the portable environment.
