An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and les...An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and less invasive surgical workflow.However,such an algorithm considers intact femur geometry only.The bone surface modification is inevitable due to intra-operative intervention.The mismatched correspondences will degrade the reliability of registered target pose.To solve this problem,this work proposed a supervised deep neural network to automatically restore the surface of processed bone.The network was trained on a synthetic dataset that consists of real depth captures of a model leg and simulated realistic femur cutting.According to the evaluation on both synthetic data and real-time captures,the registration quality can be effectively improved by surface reconstruction.The improvement in tracking accuracy is only evident over test data,indicating the need for future enhancement of the dataset and network.展开更多
The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Ki...The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.展开更多
Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation ...Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation of the user’s viewpoint(or that of a camera)with regard to the virtual content’s coordinate sys-tem.Therefore,the real-time establishment of 3-dimension(3D)maps in real scenes is particularly important for augmented reality technology.So in this paper,we integrate Simultaneous Localization and Mapping(SLAM)technology into augmented reality.Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm.In this paper we propose an improved method for Oriented FAST and Rotated BRIEF(ORB)feature extraction and optimized key frame selection,as well as the use of the Progressive Sample Consensus(PROSAC)algorithm for planar estimation of augmented reality implementations,thus solving the problem of increased sys-tem runtime because of the loss of large amounts of texture information in images.In this paper,we get better results by comparing experiments and data analysis.However,there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.展开更多
During the 2025 Beijing Digital Economy Experience Week from 27 June to 5 July,interactive projects combining technology and culture-illustrated books created by voice AI,augmented reality tours,and markerless motion ...During the 2025 Beijing Digital Economy Experience Week from 27 June to 5 July,interactive projects combining technology and culture-illustrated books created by voice AI,augmented reality tours,and markerless motion capture-attracted many visitors.“More than a dozen themed areas offered the opportunity to dive into new worlds and discover the latest innovations from more than 50 companies,”Lu Yumin,a Beijing resident who visited the event,told ChinAfrica.展开更多
OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study com...OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study compared OP and DLC estimates of lower extremity joint angles in standing movements with those of conventional software.A total of nine healthy men participated.The trial task was to stand up from a chair.The motion was recorded by a digital camera,and the joint angles of the hip and knee joints were calculated from the video using OP,DLC,and Kinovea.To confirm reliability and validity,ICC was calculated using the Kinovea value as the validity criterion and the correlation coefficient between OP and DLC.In addition,the agreement between those data was evaluated by the Bland-Altman plot.To evaluate the accuracy of the data,root means square error(RMSE)was calculated and compared for each joint.Although the correlation coefficients and ICC(2,1)were in almost perfect agreement,fixed and proportional errors were found for most joints.The RMSE was smaller for OP than for DLC.Compared to Kinovea,OP and DLC can estimate the joint angles of the hip and knee joints during the stand-up movement with an estimation error of fewer than 10,but since they are affected by the resolution of the analysis video and other factors,they need to be validated in a variety of environments and with a variety of movements.展开更多
Tracking registration is a key issue in augmented reality applications,particularly where there are no artificial identifier placed manually.In this paper,an efficient markerless tracking registration algorithm which ...Tracking registration is a key issue in augmented reality applications,particularly where there are no artificial identifier placed manually.In this paper,an efficient markerless tracking registration algorithm which combines the detector and the tracker is presented for the augmented reality system.We capture the target images in real scenes as template images,use the random ferns classifier for target detection and solve the problem of reinitialization after tracking registration failures due to changes in ambient lighting or occlusion of targets.Once the target has been successfully detected,the pyramid Lucas-Kanade(LK)optical flow tracker is used to track the detected target in real time to solve the problem of slow speed.The least median of squares(LMedS)method is used to adaptively calculate the homography matrix,and then the three-dimensional pose is estimated and the virtual object is rendered and registered.Experimental results demonstrate that the algorithm is more accurate,faster and more robust.展开更多
In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking ...In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed.The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective.Then,we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption.A point cloud-based method is used to calculate the precise pose of the camera in real time.In order to solve the problem that the traditional iterative closest point(ICP)algorithm cannot meet the real-time requirements of the system,Kdtree(k-dimensional tree)is used under the graphics processing unit(GPU)to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration.At the same time,the random sample consensus(RANSAC)algorithm is used to remove the error point pairs to improve the accuracy of the algorithm.The results show that the proposed tracking registration method has good real-time performance and robustness.展开更多
The plasmid-expression system is routinely plagued by potential plasmid instability. Chromosomal integration is one powerful approach to overcome the problem. Herein we report a plasmid-free hyper-producer E.coli stra...The plasmid-expression system is routinely plagued by potential plasmid instability. Chromosomal integration is one powerful approach to overcome the problem. Herein we report a plasmid-free hyper-producer E.coli strain for coenzyme Q10 production. A series of integration expression vectors, pxKC3T5b and pxKT5b, were constructed for chemically inducible chromosomal evolution(multiple copy integration) and replicon-free and markerless chromosomal integration(single copy integration), respectively. A coenzyme Q10 hyper-producer Escherichia coli TBW20134 was constructed by applying chemically inducible chromosomal evolution,replicon-free and markerless chromosomal integration as well as deletion of menaquinone biosynthetic pathway.The engineered E. coli TBW20134 produced 10.7 mg per gram of dry cell mass(DCM) of coenzyme Q10 when supplemented with 0.075 g·L-1of 4-hydroxy benzoic acid; this yield is unprecedented in E. coli and close to that of the commercial producer Agrobacterium tumefaciens. With this strain, the coenzyme Q10 production capacity was very stable after 30 sequential transfers and no antibiotics were required during the fermentation process. The strategy presented may be useful as a general approach for construction of stable production strains synthesizing natural products where various copy numbers for different genes are concerned.展开更多
Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essentia...Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essential role in respiratory motion management and real-time tumor tracking in external beam radiation therapy. The objective of this research is the prediction of dynamic time-series images considering the motion and the deformation of the tumor and to compensate the delay that occurs between the motion of the tumor and the beam delivery. For this, we propose a prediction algorithm for dynamic time-series images. Prediction is performed using principal component analysis (PCA) and multi-channel singular spectral analysis (MSSA). Using PCA, the motion can be denoted as a vector function and it can be estimated by its principal component which is the linear combination of eigen vectors corresponding to the largest eigen values. Time-series set of 320-detector-row CT images from lung cancer patient and kilovolt (kV) fluoroscopic images from a moving phantom were used for the evaluation of the algorithm, and both image sets were successfully predicted by the proposed algorithm. The accuracy of prediction was quite high, more than 0.999 for CT images, whereas 0.995 for kV fluoroscopic images in cross-correlation coefficient value. This algorithm for image prediction makes it possible to predict the tumor images over the next breathing period with significant accuracy.展开更多
文摘An automatic markerless knee tracking and registration algorithm has been proposed in the literature to avoid the marker insertion required by conventional computer-assisted knee surgery,resulting in a shorter and less invasive surgical workflow.However,such an algorithm considers intact femur geometry only.The bone surface modification is inevitable due to intra-operative intervention.The mismatched correspondences will degrade the reliability of registered target pose.To solve this problem,this work proposed a supervised deep neural network to automatically restore the surface of processed bone.The network was trained on a synthetic dataset that consists of real depth captures of a model leg and simulated realistic femur cutting.According to the evaluation on both synthetic data and real-time captures,the registration quality can be effectively improved by surface reconstruction.The improvement in tracking accuracy is only evident over test data,indicating the need for future enhancement of the dataset and network.
文摘The aim of this study is to propose a novel system that has an ability to detect intra-fractional motion during radiotherapy treatment in real-time using three-dimensional surface taken by a depth camera, Microsoft Kinect v1. Our approach introduces three new aspects for three-dimensional surface tracking in radiotherapy treatment. The first aspect is a new algorithm for noise reduction of depth values. Ueda’s algorithm was implemented and enabling a fast least square regression of depth values. The second aspect is an application for detection of patient’s motion at multiple points in thracoabdominal regions. The third aspect is an estimation of three-dimensional surface from multiple depth values. For evaluation of noise reduction by Ueda’s algorithm, two respiratory patterns are measured by the Kinect as well as a laser range meter. The resulting cross correlation coefficients between the laser range meter and the Kinect were 0.982 for abdominal respiration and 0.995 for breath holding. Moreover, the mean cross correlation coefficients between the signals of our system and the signals of Anzai with respect to participant’s respiratory motion were 0.90 for thoracic respiration and 0.93 for abdominal respiration, respectively. These results proved that the performance of the developed system was comparable to existing motion monitoring devices. Reconstruction of three-dimensional surface also enabled us to detect the irregular motion and breathing arrest by comparing the averaged depth with predefined threshold values.
基金supported by the Hainan Provincial Natural Science Foundation of China(project number:621QN269)the Sanya Science and Information Bureau Foundation(project number:2021GXYL251).
文摘Augmented Reality(AR)tries to seamlessly integrate virtual content into the real world of the user.Ideally,the virtual content would behave exactly like real objects.This necessitates a correct and precise estimation of the user’s viewpoint(or that of a camera)with regard to the virtual content’s coordinate sys-tem.Therefore,the real-time establishment of 3-dimension(3D)maps in real scenes is particularly important for augmented reality technology.So in this paper,we integrate Simultaneous Localization and Mapping(SLAM)technology into augmented reality.Our research is to implement an augmented reality system without markers using the ORB-SLAM2 framework algorithm.In this paper we propose an improved method for Oriented FAST and Rotated BRIEF(ORB)feature extraction and optimized key frame selection,as well as the use of the Progressive Sample Consensus(PROSAC)algorithm for planar estimation of augmented reality implementations,thus solving the problem of increased sys-tem runtime because of the loss of large amounts of texture information in images.In this paper,we get better results by comparing experiments and data analysis.However,there are some improved methods of PROSAC algorithm which are more suitable for the detection of plane feature points.
文摘During the 2025 Beijing Digital Economy Experience Week from 27 June to 5 July,interactive projects combining technology and culture-illustrated books created by voice AI,augmented reality tours,and markerless motion capture-attracted many visitors.“More than a dozen themed areas offered the opportunity to dive into new worlds and discover the latest innovations from more than 50 companies,”Lu Yumin,a Beijing resident who visited the event,told ChinAfrica.
文摘OpenPose(OP)and DeepLabCut(DLC)are applications that use deep learning to estimate posture,but there are few reports on the reliability,validity,and accuracy of their 2D lower limb joint motion analysis.This study compared OP and DLC estimates of lower extremity joint angles in standing movements with those of conventional software.A total of nine healthy men participated.The trial task was to stand up from a chair.The motion was recorded by a digital camera,and the joint angles of the hip and knee joints were calculated from the video using OP,DLC,and Kinovea.To confirm reliability and validity,ICC was calculated using the Kinovea value as the validity criterion and the correlation coefficient between OP and DLC.In addition,the agreement between those data was evaluated by the Bland-Altman plot.To evaluate the accuracy of the data,root means square error(RMSE)was calculated and compared for each joint.Although the correlation coefficients and ICC(2,1)were in almost perfect agreement,fixed and proportional errors were found for most joints.The RMSE was smaller for OP than for DLC.Compared to Kinovea,OP and DLC can estimate the joint angles of the hip and knee joints during the stand-up movement with an estimation error of fewer than 10,but since they are affected by the resolution of the analysis video and other factors,they need to be validated in a variety of environments and with a variety of movements.
基金supported by National Natural Science Foundation of China(No.61125101).
文摘Tracking registration is a key issue in augmented reality applications,particularly where there are no artificial identifier placed manually.In this paper,an efficient markerless tracking registration algorithm which combines the detector and the tracker is presented for the augmented reality system.We capture the target images in real scenes as template images,use the random ferns classifier for target detection and solve the problem of reinitialization after tracking registration failures due to changes in ambient lighting or occlusion of targets.Once the target has been successfully detected,the pyramid Lucas-Kanade(LK)optical flow tracker is used to track the detected target in real time to solve the problem of slow speed.The least median of squares(LMedS)method is used to adaptively calculate the homography matrix,and then the three-dimensional pose is estimated and the virtual object is rendered and registered.Experimental results demonstrate that the algorithm is more accurate,faster and more robust.
基金This work was supported by National Natural Science Foundation of China(No.61125101).
文摘In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed.The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective.Then,we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption.A point cloud-based method is used to calculate the precise pose of the camera in real time.In order to solve the problem that the traditional iterative closest point(ICP)algorithm cannot meet the real-time requirements of the system,Kdtree(k-dimensional tree)is used under the graphics processing unit(GPU)to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration.At the same time,the random sample consensus(RANSAC)algorithm is used to remove the error point pairs to improve the accuracy of the algorithm.The results show that the proposed tracking registration method has good real-time performance and robustness.
基金Supported by the National Natural Science Foundation of China(30970089,20876181,21276289)the Natural Science Foundation of Guangdong Province(9351027501000003,S2011010001396)
文摘The plasmid-expression system is routinely plagued by potential plasmid instability. Chromosomal integration is one powerful approach to overcome the problem. Herein we report a plasmid-free hyper-producer E.coli strain for coenzyme Q10 production. A series of integration expression vectors, pxKC3T5b and pxKT5b, were constructed for chemically inducible chromosomal evolution(multiple copy integration) and replicon-free and markerless chromosomal integration(single copy integration), respectively. A coenzyme Q10 hyper-producer Escherichia coli TBW20134 was constructed by applying chemically inducible chromosomal evolution,replicon-free and markerless chromosomal integration as well as deletion of menaquinone biosynthetic pathway.The engineered E. coli TBW20134 produced 10.7 mg per gram of dry cell mass(DCM) of coenzyme Q10 when supplemented with 0.075 g·L-1of 4-hydroxy benzoic acid; this yield is unprecedented in E. coli and close to that of the commercial producer Agrobacterium tumefaciens. With this strain, the coenzyme Q10 production capacity was very stable after 30 sequential transfers and no antibiotics were required during the fermentation process. The strategy presented may be useful as a general approach for construction of stable production strains synthesizing natural products where various copy numbers for different genes are concerned.
文摘Respiratory motion induces the limit in delivery accuracy due to the lack of the consideration of the anatomy motion in the treatment planning. Therefore, image-guided radiation therapy (IGRT) system plays an essential role in respiratory motion management and real-time tumor tracking in external beam radiation therapy. The objective of this research is the prediction of dynamic time-series images considering the motion and the deformation of the tumor and to compensate the delay that occurs between the motion of the tumor and the beam delivery. For this, we propose a prediction algorithm for dynamic time-series images. Prediction is performed using principal component analysis (PCA) and multi-channel singular spectral analysis (MSSA). Using PCA, the motion can be denoted as a vector function and it can be estimated by its principal component which is the linear combination of eigen vectors corresponding to the largest eigen values. Time-series set of 320-detector-row CT images from lung cancer patient and kilovolt (kV) fluoroscopic images from a moving phantom were used for the evaluation of the algorithm, and both image sets were successfully predicted by the proposed algorithm. The accuracy of prediction was quite high, more than 0.999 for CT images, whereas 0.995 for kV fluoroscopic images in cross-correlation coefficient value. This algorithm for image prediction makes it possible to predict the tumor images over the next breathing period with significant accuracy.
