In this paper,an integrated guidance and control approach is presented to improve the performance of the missile interception.The approach includes damping augmented system with attitude rate feedback to decrease the ...In this paper,an integrated guidance and control approach is presented to improve the performance of the missile interception.The approach includes damping augmented system with attitude rate feedback to decrease the oscillation during the homing phase for missiles with low damping.In addition,physical constraints,which can affect the performance of the missile interception,such as acceleration limit,seeker’s look angle,and look angle rate constraints are considered.The integrated guidance and control problem is formulated as a convex quadratic optimization problem with equality and inequality constraints,and the solution is obtained by a primal–dual interior point method.The performance of the proposed method is verified through several numerical examples.展开更多
The successive overrelaxation-like （SOR-like） method with the real param- eters ω is considered for solving the augmented system. The new method is called the modified SOR-like （MSOR-like） method. The functional ...The successive overrelaxation-like （SOR-like） method with the real param- eters ω is considered for solving the augmented system. The new method is called the modified SOR-like （MSOR-like） method. The functional equation between the parameters and the eigenvalues of the iteration matrix of the MSOR-like method is given. Therefore, the necessary and sufficient condition for the convergence of the MSOR-like method is derived. The optimal iteration parameter ω of the MSOR-like method is derived. Finally, the proof of theorem and numerical computation based on a particular linear system are given, which clearly show that the MSOR-like method outperforms the SOR-like （Li, C. J., Li, B. J., and Evans, D. J. Optimum accelerated parameter for the GSOR method. Neural, Parallel ＆ Scientific Computations, 7（4）, 453-462 （1999）） and the modified sym- metric SOR-like （MSSOR-like） methods （Wu, S. L., Huang, T. Z., and Zhao, X. L. A modified SSOR iterative method for augmented systems. Journal of Computational and Applied Mathematics, 228（4）, 424-433 （2009））.展开更多
Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to...Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to visualize the optic nerve intraoperatively.To address this issue,an endoscopic image-based augmented reality surgical navigation system is developed in this study.The system aims to virtually fuse the optic nerve onto the endoscopic images,assisting surgeons in determining the optic nerve’s position and reducing surgical risks.First,a calibration algorithm based on a checkerboard grid of immobile points is proposed,building upon existing calibration methods.Additionally,to tackle accuracy issues associated with augmented reality technology,an optical navigation and visual fusion compensation algorithm is proposed to improve the intraoperative tracking accuracy.To evaluate the system’s performance,model experiments were meticulously designed and conducted.The results confirm the accuracy and stability of the proposed system,with an average tracking error of(0.99±0.46)mm.This outcome demonstrates the effectiveness of the proposed algorithm in improving the augmented reality surgical navigation system’s accuracy.Furthermore,the system successfully displays hidden optic nerves and other deep tissues,thus showcasing the promising potential for future applications in orbital and maxillofacial surgery.展开更多
Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of s...Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of surgical failure.Therefore,it is necessary to design a visualization system for closed thoracic drainage.Augmented reality(AR)technology can assist in visualizing the internal anatomical structure and determining the insertion point on the body surface.The structure of the currently used steel-needle-guided chest tube was modified by integrating it with an ultrafine diameter camera to provide real-time visualization of the puncture process.After simulation experiments,the overall registration error of the AR method was measured to be within(3.59±0.53)mm,indicating its potential for clinical application.The ultrafine diameter camera module and improved steel-needle-guided chest tube can timely reflect the position of the needle tip in the human body.A comparative experiment showed that video guidance could improve the safety of the puncture process compared to the traditional method.Finally,a qualitative evaluation of the usability of the system was conducted through a questionnaire.This system facilitates the visualization of closed thoracic drainage puncture procedure and pro-vides an implementation scheme to enhance the accuracy and safety of the operative step,which is conducive to reducing the learning curve and improving the proficiency of the doctors.展开更多
In this paper, the synchronization of the fractional-order generalized augmented Lti system is investigated. Based on the predictor--corrector method, we obtain phase portraits, bifurcation diagrams, Lyapunov exponent...In this paper, the synchronization of the fractional-order generalized augmented Lti system is investigated. Based on the predictor--corrector method, we obtain phase portraits, bifurcation diagrams, Lyapunov exponent spectra, and Poincar6 maps of the fractional-order system and find that a four-wing chaotic attractor exists in the system when the system pa- rameters change within certain ranges. Further, by varying the system parameters, rich dynamical behaviors occur in the 2.7-order system. According to the stability theory of a fractional-order linear system, and adopting the linearization by feedback method, we have designed a nonlinear feedback controller in our theoretical analysis to implement the synchro- nization of the drive system with the response system. In addition, the synchronization is also shown by an electronic circuit implementation for the 2.7-order system. The obtained experiment results accord with the theoretical analyses, which further demonstrate the feasibility and effectiveness of the proposed synchronization scheme.展开更多
Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network ...Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network is prone to getting into local extrema and convergence is slow, genetic algorithm is employed to optimize the initial weights and threshold of neural network. This paper discusses how to set the crucial parameters in the algorithm. Experimental results show that the method ensures that the neural network achieves global convergence quickly and correctly. Tracking precision of AR system is improved after the tracker is rectified, and the third dimension of AR system is enhanced.展开更多
BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolvi...BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.展开更多
Background Compared with traditional thoracotomy,video-assisted thoracoscopic surgery(VATS)has less minor trauma,faster recovery,higher patient compliance,but higher requirements for surgeons.Virtual surgery training ...Background Compared with traditional thoracotomy,video-assisted thoracoscopic surgery(VATS)has less minor trauma,faster recovery,higher patient compliance,but higher requirements for surgeons.Virtual surgery training simulation systems are important and have been widely used in Europe and America.Augmented reality(AR)in surgical training simulation systems significantly improve the training effect of virtual surgical training,although AR technology is still in its initial stage.Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice.Methods This study proposed an immersive AR lobectomy within a thoracoscope surgery training system,using visual and haptic modeling to study the potential benefits of this critical technology.The content included immersive AR visual rendering,based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling.Furthermore,we designed an AR haptic rendering systems,whose model architecture consisted of multi-touch interaction points,including kinesthetic and pressure-sensitive points.Finally,based on the above theoretical research,we developed an AR interactive VATS surgical training platform.Results Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system.Face,content,and construct validation methods were used to assess the tactile sense,visual sense,scene authenticity,and simulator performance.Conclusions The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time.The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.展开更多
Background A large number of robots have put forward the new requirements for human robot interaction.One of the problems in human-swarm robot interaction is how to naturally achieve an efficient and accurate interact...Background A large number of robots have put forward the new requirements for human robot interaction.One of the problems in human-swarm robot interaction is how to naturally achieve an efficient and accurate interaction between humans and swarm robot systems.To address this,this paper proposes a new type of human-swarm natural interaction system.Methods Through the cooperation between three-dimensional(3D)gesture interaction channel and natural language instruction channel,a natural and efficient interaction between a human and swarm robots is achieved.Results First,A 3D lasso technology realizes a batch-picking interaction of swarm robots through oriented bounding boxes.Second,control instruction labels for swarm-oriented robots are defined.The instruction label is integrated with the 3D gesture and natural language through instruction label filling.Finally,the understanding of natural language instructions is realized through a text classifier based on the maximum entropy model.A head-mounted augmented reality display device is used as a visual feedback channel.Conclusions The experiments on selecting robots verify the feasibility and availability of the system.展开更多
In this paper,we present an algorithm for capacity optimization in intelligent reflecting surface(IRS)-based multiple-input multiple-output(MIMO)communication systems.To maximize the capacity of elements in IRS,we use...In this paper,we present an algorithm for capacity optimization in intelligent reflecting surface(IRS)-based multiple-input multiple-output(MIMO)communication systems.To maximize the capacity of elements in IRS,we use augmented Lagrange method with the equivalent transformations on the covariance matrix and reflection matrix constraints.This results an adjustable phase shift on the incident signal.Furthermore,we reshape the complex-valued covariance matrix and reflection matrix to a vector for the ease of calculating partial derivatives to find the search direction.Then,the quasi-Newton updates and modified Broyden-Fletcher-Goldfarb-Shano(BFGS)method in the complex domain form are used to find the local minimum.Finally,numerical simulation results demonstrate that our proposed IRS-aided system using the algorithm performs better than the state-of-the-art and the conventional communication systems.展开更多
Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal i...Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal interventions for cancer treatment.Methods:A specialized software transforms 2D medical images into precise 3D digital models, facilitating improved anatomical understanding and surgical planning. Patient-specific 3D printed anatomical models are utilized for preoperative planning, intraoperative guidance, and surgical education. AR technology enables the overlay of digital perceptions onto real-world surgical environments.Results:Patient-specific 3D printed anatomical models have multiple applications, such as preoperative planning, intraoperative guidance, trainee education, and patient counseling. Virtual reality involves substituting the real world with a computer-generated 3D environment, while AR overlays digitally created perceptions onto the existing reality. The advances in 3D modeling technology have sparked considerable interest in their application to partial nephrectomy in the realm of renal cancer. 3D printing, also known as additive manufacturing, constructs 3D objects based on computer-aided design or digital 3D models. Utilizing 3D-printed preoperative renal models provides benefits for surgical planning, offering a more reliable assessment of the tumor's relationship with vital anatomical structures and enabling better preparation for procedures. AR technology allows surgeons to visualize patient-specific renal anatomical structures and their spatial relationships with surrounding organs by projecting CT/MRI images onto a live laparoscopic video. Incorporating patient-specific 3D digital models into healthcare enhances best practice, resulting in improved patient care, increased patient satisfaction, and cost saving for the healthcare system.展开更多
This paper presents a detailed review of the applications of augmented reality(AR)in three important fields where AR use is currently increasing.The objective of this study is to highlight how AR improves and enhances...This paper presents a detailed review of the applications of augmented reality(AR)in three important fields where AR use is currently increasing.The objective of this study is to highlight how AR improves and enhances the user experience in entertainment,medicine,and retail.The authors briefly introduce the topic of AR and discuss its differences from virtual reality.They also explain the software and hardware technologies required for implementing an AR system and the different types of displays required for enhancing the user experience.The growth of AR in markets is also briefly discussed.In the three sections of the paper,the applications of AR are discussed.The use of AR in multiplayer gaming,computer games,broadcasting,and multimedia videos,as an aspect of entertainment and gaming is highlighted.AR in medicine involves the use of AR in medical healing,medical training,medical teaching,surgery,and post-medical treatment.AR in retail was discussed in terms of its uses in advertisement,marketing,fashion retail,and online shopping.The authors concluded the paper by detailing the future use of AR and its advantages and disadvantages in the current scenario.展开更多
This paper proposes an outdoor guide system using vision-based augmented reality(AR) on mobile devices.Augmented reality provides a virtual-real fusion display interface for outdoor guide.Vision-based methods are more...This paper proposes an outdoor guide system using vision-based augmented reality(AR) on mobile devices.Augmented reality provides a virtual-real fusion display interface for outdoor guide.Vision-based methods are more accurate than GPS or other hardware-based methods.However,vision-based methods require more resources and relatively strong computing power of mobile devices.A C/S framework for vision based augmented reality system is introduced in this paper.In a server,a vocabulary tree is used for location recognition.In a mobile device,BRISK feature is combined with optical flow methods to track the offline keyframe.The system is tested on UKbench datasets and in real environment.Experimental results show that the proposed vision-based augmented reality system works well and yields relatively high recognition rate and that the mobile device achieves realtime recognition performance.展开更多
Although constraint satisfaction approaches have achieved fruitful results,system states may lose their smoothness and there may be undesired chattering of control inputs due to switching characteristics.Furthermore,i...Although constraint satisfaction approaches have achieved fruitful results,system states may lose their smoothness and there may be undesired chattering of control inputs due to switching characteristics.Furthermore,it remains a challenge when there are additional constraints on control torques of robotic systems.In this article,we propose a novel high-order control barrier function(HoCBF)-based safety control method for robotic systems subject to input-output constraints,which can maintain the desired smoothness of system states and reduce undesired chattering vibration in the control torque.In our design,augmented dynamics are introduced into the HoCBF by constructing its output as the control input of the robotic system,so that the constraint satisfaction is facilitated by HoCBFs and the smoothness of system states is maintained by the augmented dynamics.This proposed scheme leads to the quadratic program(QP),which is more user-friendly in implementation since the constraint satisfaction control design is implemented as an add-on to an existing tracking control law.The proposed closed-loop control system not only achieves the requirements of real-time capability,stability,safety and compliance,but also reduces undesired chattering of control inputs.Finally,the effectiveness of the proposed control scheme is verified by simulations and experiments on robotic manipulators.展开更多
Photonic platforms are gradually emerging as a promising option to encounter the ever-growing demand for artificial intelligence,among which photonic time-delay reservoir computing(TDRC)is widely anticipated.While suc...Photonic platforms are gradually emerging as a promising option to encounter the ever-growing demand for artificial intelligence,among which photonic time-delay reservoir computing(TDRC)is widely anticipated.While such a computing paradigm can only employ a single photonic device as the nonlinear node for data processing,the performance highly relies on the fading memory provided by the delay feedback loop(FL),which sets a restriction on the extensibility of physical implementation,especially for highly integrated chips.Here,we present a simplified photonic scheme for more flexible parameter configurations leveraging the designed quasi-convolution coding(QC),which completely gets rid of the dependence on FL.Unlike delay-based TDRC,encoded data in QC-based RC(QRC)enables temporal feature extraction,facilitating augmented memory capabilities.Thus,our proposed QRC is enabled to deal with time-related tasks or sequential data without the implementation of FL.Furthermore,we can implement this hardware with a low-power,easily integrable vertical-cavity surface-emitting laser for high-performance parallel processing.We illustrate the concept validation through simulation and experimental comparison of QRC and TDRC,wherein the simpler-structured QRC outperforms across various benchmark tasks.Our results may underscore an auspicious solution for the hardware implementation of deep neural networks.展开更多
With the development of computation technology,the augmented reality(AR)is widely applied in many fields as well as the image recognition.However,the AR application on mobile platform is not developed enough in the pa...With the development of computation technology,the augmented reality(AR)is widely applied in many fields as well as the image recognition.However,the AR application on mobile platform is not developed enough in the past decades due to the capability of the mobile processors.In recent years,the performance of mobile processors has changed rapidly,which makes it comparable to the desktop processors.This paper proposed and realized an AR system to be used on the Android mobile platform based on the image recognition through EasyAR engine and Unity 3D development tools.In this system,the image recognition could be done locally and/or using cloud recognition.Test results show that the cloud-based recognition is more efficient and accuracy than the local recognition for the mobile AR when there are more images to be recognized at the same time.展开更多
Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in ed...Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in education continues to increase,educators actively seek innovative and immersive methods to engage students in learning.However,exploring these possibilities also entails identifying and overcoming existing barriers to optimal educational integration.Concurrently,this surge in demand has prompted the identification of specific barriers,one of which is three-dimensional(3D)modeling.Creating 3D objects for augmented reality education applications can be challenging and time-consuming for the educators.To address this,we have developed a pipeline that creates realistic 3D objects from the two-dimensional(2D)photograph.Applications for augmented and virtual reality can then utilize these created 3D objects.We evaluated the proposed pipeline based on the usability of the 3D object and performance metrics.Quantitatively,with 117 respondents,the co-creation team was surveyed with openended questions to evaluate the precision of the 3D object created by the proposed photogrammetry pipeline.We analyzed the survey data using descriptive-analytical methods and found that the proposed pipeline produces 3D models that are positively accurate when compared to real-world objects,with an average mean score above 8.This study adds new knowledge in creating 3D objects for augmented reality applications by using the photogrammetry technique;finally,it discusses potential problems and future research directions for 3D objects in the education sector.展开更多
The article presents a fragment of research and development, which objective was to develop technical tools and methodology to improve exploitation processes of energy systems. The author＇s model includes synergy of ...The article presents a fragment of research and development, which objective was to develop technical tools and methodology to improve exploitation processes of energy systems. The author＇s model includes synergy of artificial intelligence and augmented reality. This solution, which combines modem technologies in order to improve the activities related to the continuity of energy supply, and reduce costs associated with the time needed to carry out exploitation activities and employment of qualified staff, is presented. This paper presents both theoretical foundations as well as the development of technical systems. The characteristics of exploitation processes of energy systems and possible technical conditions, as well as factors characterizing them, are discussed. The physical and software structures of the system and individual modules, as well as dependencies connecting them are demonstrated. The dependencies between physical and logical elements during the exploitation processes of energy systems, that determine decisions related to the evaluation of technical states and related activities are described. The advantages and limitations of the developed model which connects methods of data processing and analysis, interactive visualization processes and possible areas of application are as well discussed in detailed.展开更多
Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)t...Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)training the model solely with copy-paste mixed pictures from labeled and unlabeled input loses a lot of labeled information;(2)low-quality pseudo-labels can cause confirmation bias in pseudo-supervised learning on unlabeled data;(3)the segmentation performance in low-contrast and local regions is less than optimal.We design a Stochastic Augmentation-Based Dual-Teaching Auxiliary Training Strategy(SADT),which enhances feature diversity and learns high-quality features to overcome these problems.To be more precise,SADT trains the Student Network by using pseudo-label-based training from Teacher Network 1 and supervised learning with labeled data,which prevents the loss of rare labeled data.We introduce a bi-directional copy-pastemask with progressive high-entropy filtering to reduce data distribution disparities and mitigate confirmation bias in pseudo-supervision.For the mixed images,Deep-Shallow Spatial Contrastive Learning(DSSCL)is proposed in the feature spaces of Teacher Network 2 and the Student Network to improve the segmentation capabilities in low-contrast and local areas.In this procedure,the features retrieved by the Student Network are subjected to a random feature perturbation technique.On two openly available datasets,extensive trials show that our proposed SADT performs much better than the state-ofthe-art semi-supervised medical segmentation techniques.Using only 10%of the labeled data for training,SADT was able to acquire a Dice score of 90.10%on the ACDC(Automatic Cardiac Diagnosis Challenge)dataset.展开更多
文摘In this paper,an integrated guidance and control approach is presented to improve the performance of the missile interception.The approach includes damping augmented system with attitude rate feedback to decrease the oscillation during the homing phase for missiles with low damping.In addition,physical constraints,which can affect the performance of the missile interception,such as acceleration limit,seeker’s look angle,and look angle rate constraints are considered.The integrated guidance and control problem is formulated as a convex quadratic optimization problem with equality and inequality constraints,and the solution is obtained by a primal–dual interior point method.The performance of the proposed method is verified through several numerical examples.
基金supported by the National Natural Science Foundation of China(No.10771031)the Fundamental Research Funds for Central Universities(No.090405013)
文摘The successive overrelaxation-like （SOR-like） method with the real param- eters ω is considered for solving the augmented system. The new method is called the modified SOR-like （MSOR-like） method. The functional equation between the parameters and the eigenvalues of the iteration matrix of the MSOR-like method is given. Therefore, the necessary and sufficient condition for the convergence of the MSOR-like method is derived. The optimal iteration parameter ω of the MSOR-like method is derived. Finally, the proof of theorem and numerical computation based on a particular linear system are given, which clearly show that the MSOR-like method outperforms the SOR-like （Li, C. J., Li, B. J., and Evans, D. J. Optimum accelerated parameter for the GSOR method. Neural, Parallel ＆ Scientific Computations, 7（4）, 453-462 （1999）） and the modified sym- metric SOR-like （MSSOR-like） methods （Wu, S. L., Huang, T. Z., and Zhao, X. L. A modified SSOR iterative method for augmented systems. Journal of Computational and Applied Mathematics, 228（4）, 424-433 （2009））.
基金the National Natural Science Foundation of China(Nos.82330063 and M-0019)the Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2022QN056,YG2023ZD19,and YG2023ZD15)+2 种基金the Cross Disciplinary Research Fund of Shanghai Ninth People’s Hospital,Shanghai Jiao Tong University School of Medicine(No.JYJC202115)the Translation Clinical R&D Project of Medical Robot of Shanghai Ninth People’s Hospital,Shanghai Jiao Tong University School of Medicine(No.IMR-NPH202002)the Shanghai Key Clinical Specialty,Shanghai Eye Disease Research Center(No.2022ZZ01003)。
文摘Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to visualize the optic nerve intraoperatively.To address this issue,an endoscopic image-based augmented reality surgical navigation system is developed in this study.The system aims to virtually fuse the optic nerve onto the endoscopic images,assisting surgeons in determining the optic nerve’s position and reducing surgical risks.First,a calibration algorithm based on a checkerboard grid of immobile points is proposed,building upon existing calibration methods.Additionally,to tackle accuracy issues associated with augmented reality technology,an optical navigation and visual fusion compensation algorithm is proposed to improve the intraoperative tracking accuracy.To evaluate the system’s performance,model experiments were meticulously designed and conducted.The results confirm the accuracy and stability of the proposed system,with an average tracking error of(0.99±0.46)mm.This outcome demonstrates the effectiveness of the proposed algorithm in improving the augmented reality surgical navigation system’s accuracy.Furthermore,the system successfully displays hidden optic nerves and other deep tissues,thus showcasing the promising potential for future applications in orbital and maxillofacial surgery.
基金the Shanghai Municipal Education Commission-Gaofeng Clinical Medicine Grant(No.20172005)。
文摘Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of surgical failure.Therefore,it is necessary to design a visualization system for closed thoracic drainage.Augmented reality(AR)technology can assist in visualizing the internal anatomical structure and determining the insertion point on the body surface.The structure of the currently used steel-needle-guided chest tube was modified by integrating it with an ultrafine diameter camera to provide real-time visualization of the puncture process.After simulation experiments,the overall registration error of the AR method was measured to be within(3.59±0.53)mm,indicating its potential for clinical application.The ultrafine diameter camera module and improved steel-needle-guided chest tube can timely reflect the position of the needle tip in the human body.A comparative experiment showed that video guidance could improve the safety of the puncture process compared to the traditional method.Finally,a qualitative evaluation of the usability of the system was conducted through a questionnaire.This system facilitates the visualization of closed thoracic drainage puncture procedure and pro-vides an implementation scheme to enhance the accuracy and safety of the operative step,which is conducive to reducing the learning curve and improving the proficiency of the doctors.
基金supported by the National Natural Science Foundation of China(Grant No.61174094)the Young Scientists Fund of the National Natural Science Foundation of China(Grant No.11202148)
文摘In this paper, the synchronization of the fractional-order generalized augmented Lti system is investigated. Based on the predictor--corrector method, we obtain phase portraits, bifurcation diagrams, Lyapunov exponent spectra, and Poincar6 maps of the fractional-order system and find that a four-wing chaotic attractor exists in the system when the system pa- rameters change within certain ranges. Further, by varying the system parameters, rich dynamical behaviors occur in the 2.7-order system. According to the stability theory of a fractional-order linear system, and adopting the linearization by feedback method, we have designed a nonlinear feedback controller in our theoretical analysis to implement the synchro- nization of the drive system with the response system. In addition, the synchronization is also shown by an electronic circuit implementation for the 2.7-order system. The obtained experiment results accord with the theoretical analyses, which further demonstrate the feasibility and effectiveness of the proposed synchronization scheme.
基金Project supported by Science Foundation of Shanghai Municipal Commission of Science and Technology (Grant No .025115008)
文摘Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network is prone to getting into local extrema and convergence is slow, genetic algorithm is employed to optimize the initial weights and threshold of neural network. This paper discusses how to set the crucial parameters in the algorithm. Experimental results show that the method ensures that the neural network achieves global convergence quickly and correctly. Tracking precision of AR system is improved after the tracker is rectified, and the third dimension of AR system is enhanced.
基金supported by Tianjin Sci-tech Planning Projects (14RCGFGX00846)Natural Science Foundation of Hebei Province (F2015202239)+1 种基金Tianjin Sci-tech Planning Projects (15ZCZDNC00130)Science and Technology Research Project of Hebei Province (Z2015044)
文摘BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.
基金the National Science Foundations of China(62062069,62062070,62005235).
文摘Background Compared with traditional thoracotomy,video-assisted thoracoscopic surgery(VATS)has less minor trauma,faster recovery,higher patient compliance,but higher requirements for surgeons.Virtual surgery training simulation systems are important and have been widely used in Europe and America.Augmented reality(AR)in surgical training simulation systems significantly improve the training effect of virtual surgical training,although AR technology is still in its initial stage.Mixed reality has gained increased attention in technology-driven modern medicine but has yet to be used in everyday practice.Methods This study proposed an immersive AR lobectomy within a thoracoscope surgery training system,using visual and haptic modeling to study the potential benefits of this critical technology.The content included immersive AR visual rendering,based on the cluster-based extended position-based dynamics algorithm of soft tissue physical modeling.Furthermore,we designed an AR haptic rendering systems,whose model architecture consisted of multi-touch interaction points,including kinesthetic and pressure-sensitive points.Finally,based on the above theoretical research,we developed an AR interactive VATS surgical training platform.Results Twenty-four volunteers were recruited from the First People's Hospital of Yunnan Province to evaluate the VATS training system.Face,content,and construct validation methods were used to assess the tactile sense,visual sense,scene authenticity,and simulator performance.Conclusions The results of our construction validation demonstrate that the simulator is useful in improving novice and surgical skills that can be retained after a certain period of time.The video-assisted thoracoscopic system based on AR developed in this study is effective and can be used as a training device to assist in the development of thoracoscopic skills for novices.
基金Key-Area Research and Development Program of Guangdong Province(2019B090915002).
文摘Background A large number of robots have put forward the new requirements for human robot interaction.One of the problems in human-swarm robot interaction is how to naturally achieve an efficient and accurate interaction between humans and swarm robot systems.To address this,this paper proposes a new type of human-swarm natural interaction system.Methods Through the cooperation between three-dimensional(3D)gesture interaction channel and natural language instruction channel,a natural and efficient interaction between a human and swarm robots is achieved.Results First,A 3D lasso technology realizes a batch-picking interaction of swarm robots through oriented bounding boxes.Second,control instruction labels for swarm-oriented robots are defined.The instruction label is integrated with the 3D gesture and natural language through instruction label filling.Finally,the understanding of natural language instructions is realized through a text classifier based on the maximum entropy model.A head-mounted augmented reality display device is used as a visual feedback channel.Conclusions The experiments on selecting robots verify the feasibility and availability of the system.
基金This work was supported by the National Natural Science Foundation of China(No.61771248,61971167,62001056)Startup Foundation for Introducing Talent of NUIST,and Open Research Fund of National Mobile Communications Research Laboratory,Southeast University(No.2020D14).
文摘In this paper,we present an algorithm for capacity optimization in intelligent reflecting surface(IRS)-based multiple-input multiple-output(MIMO)communication systems.To maximize the capacity of elements in IRS,we use augmented Lagrange method with the equivalent transformations on the covariance matrix and reflection matrix constraints.This results an adjustable phase shift on the incident signal.Furthermore,we reshape the complex-valued covariance matrix and reflection matrix to a vector for the ease of calculating partial derivatives to find the search direction.Then,the quasi-Newton updates and modified Broyden-Fletcher-Goldfarb-Shano(BFGS)method in the complex domain form are used to find the local minimum.Finally,numerical simulation results demonstrate that our proposed IRS-aided system using the algorithm performs better than the state-of-the-art and the conventional communication systems.
文摘Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal interventions for cancer treatment.Methods:A specialized software transforms 2D medical images into precise 3D digital models, facilitating improved anatomical understanding and surgical planning. Patient-specific 3D printed anatomical models are utilized for preoperative planning, intraoperative guidance, and surgical education. AR technology enables the overlay of digital perceptions onto real-world surgical environments.Results:Patient-specific 3D printed anatomical models have multiple applications, such as preoperative planning, intraoperative guidance, trainee education, and patient counseling. Virtual reality involves substituting the real world with a computer-generated 3D environment, while AR overlays digitally created perceptions onto the existing reality. The advances in 3D modeling technology have sparked considerable interest in their application to partial nephrectomy in the realm of renal cancer. 3D printing, also known as additive manufacturing, constructs 3D objects based on computer-aided design or digital 3D models. Utilizing 3D-printed preoperative renal models provides benefits for surgical planning, offering a more reliable assessment of the tumor's relationship with vital anatomical structures and enabling better preparation for procedures. AR technology allows surgeons to visualize patient-specific renal anatomical structures and their spatial relationships with surrounding organs by projecting CT/MRI images onto a live laparoscopic video. Incorporating patient-specific 3D digital models into healthcare enhances best practice, resulting in improved patient care, increased patient satisfaction, and cost saving for the healthcare system.
文摘This paper presents a detailed review of the applications of augmented reality(AR)in three important fields where AR use is currently increasing.The objective of this study is to highlight how AR improves and enhances the user experience in entertainment,medicine,and retail.The authors briefly introduce the topic of AR and discuss its differences from virtual reality.They also explain the software and hardware technologies required for implementing an AR system and the different types of displays required for enhancing the user experience.The growth of AR in markets is also briefly discussed.In the three sections of the paper,the applications of AR are discussed.The use of AR in multiplayer gaming,computer games,broadcasting,and multimedia videos,as an aspect of entertainment and gaming is highlighted.AR in medicine involves the use of AR in medical healing,medical training,medical teaching,surgery,and post-medical treatment.AR in retail was discussed in terms of its uses in advertisement,marketing,fashion retail,and online shopping.The authors concluded the paper by detailing the future use of AR and its advantages and disadvantages in the current scenario.
基金Supported by the National Science and Technology Major Project(No.2012ZX03002004)the National High Technology Research and Development Programme of China(No.2013AA013802)
文摘This paper proposes an outdoor guide system using vision-based augmented reality(AR) on mobile devices.Augmented reality provides a virtual-real fusion display interface for outdoor guide.Vision-based methods are more accurate than GPS or other hardware-based methods.However,vision-based methods require more resources and relatively strong computing power of mobile devices.A C/S framework for vision based augmented reality system is introduced in this paper.In a server,a vocabulary tree is used for location recognition.In a mobile device,BRISK feature is combined with optical flow methods to track the offline keyframe.The system is tested on UKbench datasets and in real environment.Experimental results show that the proposed vision-based augmented reality system works well and yields relatively high recognition rate and that the mobile device achieves realtime recognition performance.
基金supported in part by the National Natural Science Foundation of China(62273311,61773351)Henan Provincial Science Foundation for Distinguished Young Scholars(242300421051)。
文摘Although constraint satisfaction approaches have achieved fruitful results,system states may lose their smoothness and there may be undesired chattering of control inputs due to switching characteristics.Furthermore,it remains a challenge when there are additional constraints on control torques of robotic systems.In this article,we propose a novel high-order control barrier function(HoCBF)-based safety control method for robotic systems subject to input-output constraints,which can maintain the desired smoothness of system states and reduce undesired chattering vibration in the control torque.In our design,augmented dynamics are introduced into the HoCBF by constructing its output as the control input of the robotic system,so that the constraint satisfaction is facilitated by HoCBFs and the smoothness of system states is maintained by the augmented dynamics.This proposed scheme leads to the quadratic program(QP),which is more user-friendly in implementation since the constraint satisfaction control design is implemented as an add-on to an existing tracking control law.The proposed closed-loop control system not only achieves the requirements of real-time capability,stability,safety and compliance,but also reduces undesired chattering of control inputs.Finally,the effectiveness of the proposed control scheme is verified by simulations and experiments on robotic manipulators.
基金National Natural Science Foundation of China(62171305,62405206,62004135,62001317,62111530301)Natural Science Foundation of Jiangsu Province(BK20240778,BK20241917)+3 种基金State Key Laboratory of Advanced Optical Communication Systems and Networks,China(2023GZKF08)China Postdoctoral Science Foundation(2024M752314)Postdoctoral Fellowship Program of CPSF(GZC20231883)Innovative and Entrepreneurial Talent Program of Jiangsu Province(JSSCRC2021527).
文摘Photonic platforms are gradually emerging as a promising option to encounter the ever-growing demand for artificial intelligence,among which photonic time-delay reservoir computing(TDRC)is widely anticipated.While such a computing paradigm can only employ a single photonic device as the nonlinear node for data processing,the performance highly relies on the fading memory provided by the delay feedback loop(FL),which sets a restriction on the extensibility of physical implementation,especially for highly integrated chips.Here,we present a simplified photonic scheme for more flexible parameter configurations leveraging the designed quasi-convolution coding(QC),which completely gets rid of the dependence on FL.Unlike delay-based TDRC,encoded data in QC-based RC(QRC)enables temporal feature extraction,facilitating augmented memory capabilities.Thus,our proposed QRC is enabled to deal with time-related tasks or sequential data without the implementation of FL.Furthermore,we can implement this hardware with a low-power,easily integrable vertical-cavity surface-emitting laser for high-performance parallel processing.We illustrate the concept validation through simulation and experimental comparison of QRC and TDRC,wherein the simpler-structured QRC outperforms across various benchmark tasks.Our results may underscore an auspicious solution for the hardware implementation of deep neural networks.
文摘With the development of computation technology,the augmented reality(AR)is widely applied in many fields as well as the image recognition.However,the AR application on mobile platform is not developed enough in the past decades due to the capability of the mobile processors.In recent years,the performance of mobile processors has changed rapidly,which makes it comparable to the desktop processors.This paper proposed and realized an AR system to be used on the Android mobile platform based on the image recognition through EasyAR engine and Unity 3D development tools.In this system,the image recognition could be done locally and/or using cloud recognition.Test results show that the cloud-based recognition is more efficient and accuracy than the local recognition for the mobile AR when there are more images to be recognized at the same time.
文摘Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in education continues to increase,educators actively seek innovative and immersive methods to engage students in learning.However,exploring these possibilities also entails identifying and overcoming existing barriers to optimal educational integration.Concurrently,this surge in demand has prompted the identification of specific barriers,one of which is three-dimensional(3D)modeling.Creating 3D objects for augmented reality education applications can be challenging and time-consuming for the educators.To address this,we have developed a pipeline that creates realistic 3D objects from the two-dimensional(2D)photograph.Applications for augmented and virtual reality can then utilize these created 3D objects.We evaluated the proposed pipeline based on the usability of the 3D object and performance metrics.Quantitatively,with 117 respondents,the co-creation team was surveyed with openended questions to evaluate the precision of the 3D object created by the proposed photogrammetry pipeline.We analyzed the survey data using descriptive-analytical methods and found that the proposed pipeline produces 3D models that are positively accurate when compared to real-world objects,with an average mean score above 8.This study adds new knowledge in creating 3D objects for augmented reality applications by using the photogrammetry technique;finally,it discusses potential problems and future research directions for 3D objects in the education sector.
文摘The article presents a fragment of research and development, which objective was to develop technical tools and methodology to improve exploitation processes of energy systems. The author＇s model includes synergy of artificial intelligence and augmented reality. This solution, which combines modem technologies in order to improve the activities related to the continuity of energy supply, and reduce costs associated with the time needed to carry out exploitation activities and employment of qualified staff, is presented. This paper presents both theoretical foundations as well as the development of technical systems. The characteristics of exploitation processes of energy systems and possible technical conditions, as well as factors characterizing them, are discussed. The physical and software structures of the system and individual modules, as well as dependencies connecting them are demonstrated. The dependencies between physical and logical elements during the exploitation processes of energy systems, that determine decisions related to the evaluation of technical states and related activities are described. The advantages and limitations of the developed model which connects methods of data processing and analysis, interactive visualization processes and possible areas of application are as well discussed in detailed.
基金supported by the Natural Science Foundation of China(No.41804112,author:Chengyun Song).
文摘Existing semi-supervisedmedical image segmentation algorithms use copy-paste data augmentation to correct the labeled-unlabeled data distribution mismatch.However,current copy-paste methods have three limitations:(1)training the model solely with copy-paste mixed pictures from labeled and unlabeled input loses a lot of labeled information;(2)low-quality pseudo-labels can cause confirmation bias in pseudo-supervised learning on unlabeled data;(3)the segmentation performance in low-contrast and local regions is less than optimal.We design a Stochastic Augmentation-Based Dual-Teaching Auxiliary Training Strategy(SADT),which enhances feature diversity and learns high-quality features to overcome these problems.To be more precise,SADT trains the Student Network by using pseudo-label-based training from Teacher Network 1 and supervised learning with labeled data,which prevents the loss of rare labeled data.We introduce a bi-directional copy-pastemask with progressive high-entropy filtering to reduce data distribution disparities and mitigate confirmation bias in pseudo-supervision.For the mixed images,Deep-Shallow Spatial Contrastive Learning(DSSCL)is proposed in the feature spaces of Teacher Network 2 and the Student Network to improve the segmentation capabilities in low-contrast and local areas.In this procedure,the features retrieved by the Student Network are subjected to a random feature perturbation technique.On two openly available datasets,extensive trials show that our proposed SADT performs much better than the state-ofthe-art semi-supervised medical segmentation techniques.Using only 10%of the labeled data for training,SADT was able to acquire a Dice score of 90.10%on the ACDC(Automatic Cardiac Diagnosis Challenge)dataset.
