Reconstructing a three-dimensional(3D)environment is an indispensable technique to make augmented reality and augmented virtuality feasible.A Kinect device is an efficient tool for reconstructing 3D environments,and u...Reconstructing a three-dimensional(3D)environment is an indispensable technique to make augmented reality and augmented virtuality feasible.A Kinect device is an efficient tool for reconstructing 3D environments,and using multiple Kinect devices enables the enhancement of reconstruction density and expansion of virtual spaces.To employ multiple devices simultaneously,Kinect devices need to be calibrated with respect to each other.There are several schemes available that calibrate 3D images generated frommultiple Kinect devices,including themarker detection method.In this study,we introduce a markerless calibration technique for Azure Kinect devices that avoids the drawbacks of marker detection,which directly affects calibration accuracy;it offers superior userfriendliness,efficiency,and accuracy.Further,we applied a joint tracking algorithm to approximate the calibration.Traditional methods require the information of multiple joints for calibration;however,Azure Kinect,the latest version of Kinect,requires the information of only one joint.The obtained result was further refined using the iterative closest point algorithm.We conducted several experimental tests that confirmed the enhanced efficiency and accuracy of the proposed method for multiple Kinect devices when compared to the conventional markerbased calibration.展开更多
Nondestructive measurement technology of phenotype can provide substantial phenotypic data support for applications such as seedling breeding,management,and quality testing.The current method of measuring seedling phe...Nondestructive measurement technology of phenotype can provide substantial phenotypic data support for applications such as seedling breeding,management,and quality testing.The current method of measuring seedling phenotypes mainly relies on manual measurement which is inefficient,subjective and destroys samples.Therefore,the paper proposes a nondestructive measurement method for the canopy phenotype of the watermelon plug seedlings based on deep learning.The Azure Kinect was used to shoot canopy color images,depth images,and RGB-D images of the watermelon plug seedlings.The Mask-RCNN network was used to classify,segment,and count the canopy leaves of the watermelon plug seedlings.To reduce the error of leaf area measurement caused by mutual occlusion of leaves,the leaves were repaired by CycleGAN,and the depth images were restored by image processing.Then,the Delaunay triangulation was adopted to measure the leaf area in the leaf point cloud.The YOLOX target detection network was used to identify the growing point position of each seedling on the plug tray.Then the depth differences between the growing point and the upper surface of the plug tray were calculated to obtain plant height.The experiment results show that the nondestructive measurement algorithm proposed in this paper achieves good measurement performance for the watermelon plug seedlings from the 1 true-leaf to 3 true-leaf stages.The average relative error of measurement is 2.33%for the number of true leaves,4.59%for the number of cotyledons,8.37%for the leaf area,and 3.27%for the plant height.The experiment results demonstrate that the proposed algorithm in this paper provides an effective solution for the nondestructive measurement of the canopy phenotype of the plug seedlings.展开更多
Timely and effective knee function evaluation and knee exercises promote the prevention and self-management of knee diseases.In this paper,aKinectbased exergame system is proposed to assess and train the knee function...Timely and effective knee function evaluation and knee exercises promote the prevention and self-management of knee diseases.In this paper,aKinectbased exergame system is proposed to assess and train the knee function.Azure Kinect was used to capture and generate 3D models of the user and immerse them in an interactive virtual environment.The software included three functional modules:knee function evaluation,Knee exercises game,and Comprehensive evaluation.The stand,step,leg lift,and squat were selected for knee function evaluation and exercises.Twenty volunteers participated in the experiment.Intraclass correlation coefficients(ICCs)were calculated to assess the reliability of kinematic measurements of knee angles during the movements.The ICC of these movements were stand(ICC=0.987),step(ICC=0.997),left leg lift(ICC=0.981),right leg lift(ICC=0.990),stand(ICC=0.998).The results show that the test-retest reliability is high.It means that the motion capture data is effective and the data obtained by Kinect is stable.展开更多
This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedes...This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedestrian detection algorithms during training. Furthermore, it is dedicated to addressing occlusion issues in pedestrian detection by using Azure Kinect for body tracking and integrating a robust occlusion management algorithm, significantly enhancing detection efficiency. In experiments, an average latency of 204 milliseconds was measured, and the detection accuracy reached an outstanding level of 97%. Additionally, this approach has been successfully applied in creating a simple yet captivating augmented reality game, demonstrating the practical application of the algorithm.展开更多
基金supported by Basic Science Research Program through the National Research Foundation of Korea(NRF)funded by the Korea Government(MSIT)(Grant No.NRF-2022R1A2C1004588).
文摘Reconstructing a three-dimensional(3D)environment is an indispensable technique to make augmented reality and augmented virtuality feasible.A Kinect device is an efficient tool for reconstructing 3D environments,and using multiple Kinect devices enables the enhancement of reconstruction density and expansion of virtual spaces.To employ multiple devices simultaneously,Kinect devices need to be calibrated with respect to each other.There are several schemes available that calibrate 3D images generated frommultiple Kinect devices,including themarker detection method.In this study,we introduce a markerless calibration technique for Azure Kinect devices that avoids the drawbacks of marker detection,which directly affects calibration accuracy;it offers superior userfriendliness,efficiency,and accuracy.Further,we applied a joint tracking algorithm to approximate the calibration.Traditional methods require the information of multiple joints for calibration;however,Azure Kinect,the latest version of Kinect,requires the information of only one joint.The obtained result was further refined using the iterative closest point algorithm.We conducted several experimental tests that confirmed the enhanced efficiency and accuracy of the proposed method for multiple Kinect devices when compared to the conventional markerbased calibration.
基金funded by the National Key Research and Development Program of China(Grant No.2019YFD1001900)the HZAU-AGIS Cooperation Fund(Grant No.SZYJY2022006).
文摘Nondestructive measurement technology of phenotype can provide substantial phenotypic data support for applications such as seedling breeding,management,and quality testing.The current method of measuring seedling phenotypes mainly relies on manual measurement which is inefficient,subjective and destroys samples.Therefore,the paper proposes a nondestructive measurement method for the canopy phenotype of the watermelon plug seedlings based on deep learning.The Azure Kinect was used to shoot canopy color images,depth images,and RGB-D images of the watermelon plug seedlings.The Mask-RCNN network was used to classify,segment,and count the canopy leaves of the watermelon plug seedlings.To reduce the error of leaf area measurement caused by mutual occlusion of leaves,the leaves were repaired by CycleGAN,and the depth images were restored by image processing.Then,the Delaunay triangulation was adopted to measure the leaf area in the leaf point cloud.The YOLOX target detection network was used to identify the growing point position of each seedling on the plug tray.Then the depth differences between the growing point and the upper surface of the plug tray were calculated to obtain plant height.The experiment results show that the nondestructive measurement algorithm proposed in this paper achieves good measurement performance for the watermelon plug seedlings from the 1 true-leaf to 3 true-leaf stages.The average relative error of measurement is 2.33%for the number of true leaves,4.59%for the number of cotyledons,8.37%for the leaf area,and 3.27%for the plant height.The experiment results demonstrate that the proposed algorithm in this paper provides an effective solution for the nondestructive measurement of the canopy phenotype of the plug seedlings.
基金This research was funded by the Key Project on Anhui Provincial Natural Science Study by Colleges and Universities under Grant“Key technical research of knee function evaluation and rehabilitation training”(No.KJ2019A0555)Key project of Science and Technology Service Network Program of Chinese Academy of Sciences“Construction of chronic disease risk prevention and control service system based on big data”(No.KFJ-STS-ZDTP-079)Major Science and Technology Projects ofAnhui Province”Research and demonstration of key technologies of non-medical sexual health promotion services”(No.18030801133).
文摘Timely and effective knee function evaluation and knee exercises promote the prevention and self-management of knee diseases.In this paper,aKinectbased exergame system is proposed to assess and train the knee function.Azure Kinect was used to capture and generate 3D models of the user and immerse them in an interactive virtual environment.The software included three functional modules:knee function evaluation,Knee exercises game,and Comprehensive evaluation.The stand,step,leg lift,and squat were selected for knee function evaluation and exercises.Twenty volunteers participated in the experiment.Intraclass correlation coefficients(ICCs)were calculated to assess the reliability of kinematic measurements of knee angles during the movements.The ICC of these movements were stand(ICC=0.987),step(ICC=0.997),left leg lift(ICC=0.981),right leg lift(ICC=0.990),stand(ICC=0.998).The results show that the test-retest reliability is high.It means that the motion capture data is effective and the data obtained by Kinect is stable.
文摘This study explores the challenges posed by pedestrian detection and occlusion in AR applications, employing a novel approach that utilizes RGB-D-based skeleton reconstruction to reduce the overhead of classical pedestrian detection algorithms during training. Furthermore, it is dedicated to addressing occlusion issues in pedestrian detection by using Azure Kinect for body tracking and integrating a robust occlusion management algorithm, significantly enhancing detection efficiency. In experiments, an average latency of 204 milliseconds was measured, and the detection accuracy reached an outstanding level of 97%. Additionally, this approach has been successfully applied in creating a simple yet captivating augmented reality game, demonstrating the practical application of the algorithm.
