Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in ed...Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in education continues to increase,educators actively seek innovative and immersive methods to engage students in learning.However,exploring these possibilities also entails identifying and overcoming existing barriers to optimal educational integration.Concurrently,this surge in demand has prompted the identification of specific barriers,one of which is three-dimensional(3D)modeling.Creating 3D objects for augmented reality education applications can be challenging and time-consuming for the educators.To address this,we have developed a pipeline that creates realistic 3D objects from the two-dimensional(2D)photograph.Applications for augmented and virtual reality can then utilize these created 3D objects.We evaluated the proposed pipeline based on the usability of the 3D object and performance metrics.Quantitatively,with 117 respondents,the co-creation team was surveyed with openended questions to evaluate the precision of the 3D object created by the proposed photogrammetry pipeline.We analyzed the survey data using descriptive-analytical methods and found that the proposed pipeline produces 3D models that are positively accurate when compared to real-world objects,with an average mean score above 8.This study adds new knowledge in creating 3D objects for augmented reality applications by using the photogrammetry technique;finally,it discusses potential problems and future research directions for 3D objects in the education sector.展开更多
The purpose of this study is to establish a multivariate nonlinear regression mathematical model to predict the displacement of tumor during brain tumor resection surgery.And the study will be integrated with augmente...The purpose of this study is to establish a multivariate nonlinear regression mathematical model to predict the displacement of tumor during brain tumor resection surgery.And the study will be integrated with augmented reality technology to achieve three-dimensional visualization,thereby enhancing the complete resection rate of tumor and the success rate of surgery.Based on the preoperative MRI data of the patients,a 3D virtual model is reconstructed and 3D printed.A brain biomimetic model is created using gel injection molding.By considering cerebrospinal fluid loss and tumor cyst fluid loss as independent variables,the highest point displacement in the vertical bone window direction is determined as the dependent variable after positioning the patient for surgery.An orthogonal experiment is conducted on the biomimetic model to establish a predictive model,and this model is incorporated into the augmented reality navigation system.To validate the predictive model,five participants wore HoloLens2 devices,overlaying the patient’s 3D virtual model onto the physical head model.Subsequently,the spatial coordinates of the tumor’s highest point after displacement were measured on both the physical and virtual models(actual coordinates and predicted coordinates,respectively).The difference between these coordinates represents the model’s prediction error.The results indicate that the measured and predicted errors for the displacement of the tumor’s highest point on the X and Y axes range from−0.6787 mm to 0.2957 mm and−0.4314 mm to 0.2253 mm,respectively.The relative errors for each experimental group are within 10%,demonstrating a good fit of the model.This method of establishing a regression model represents a preliminary attempt to predict brain tumor displacement in specific situations.It also provides a new approach for surgeons.By combining augmented reality visualization,it addresses the need for predicting tumor displacement and precisely locating brain anatomical structures in a simple and cost-effective manner.展开更多
Augmented reality(AR)is a technology that superimposes digital information onto real-world objects via head-mounted display devices to improve surgical finesse through visually enhanced medical information.With the ra...Augmented reality(AR)is a technology that superimposes digital information onto real-world objects via head-mounted display devices to improve surgical finesse through visually enhanced medical information.With the rapid development of digital technology,AR has been increasingly adopted in orthopedic surgeries across the globe,especially in total knee arthroplasty procedures which demand high precision.By overlaying digital information onto the surgeon's field of view,AR systems enhance precision,improve alignment accuracy,and reduce the risk of complications associated with malalignment.Some concerns have been raised despite accuracy,including the learning curve,long-term outcomes,and technical limitations.Furthermore,it is essential for health practitioners to gain trust in the utilisation of AR.展开更多
With the iteration and upgrading of medical technology and the continuous growth of public health demands,the quality of nursing services has become a core indicator for measuring the effectiveness of the medical syst...With the iteration and upgrading of medical technology and the continuous growth of public health demands,the quality of nursing services has become a core indicator for measuring the effectiveness of the medical system.The clinical practice ability of nursing staff is directly related to the safety of patient diagnosis and treatment and the rehabilitation process.However,the current clinical nursing talent training model is facing bottlenecks such as limited practical scenarios and fragmented case cognition.This study focuses on the teaching application of augmented reality(AR)technology in hospital Settings and systematically reviews the research progress on the improvement of clinical practice ability of trainee nurses based on the AR immersive teaching model.By constructing a clinical teaching scenario that integrates virtual and real,AR technology can dynamically simulate complex case handling processes and enhance nursing students’three-dimensional cognition of condition assessment,operation norms,and emergency plans.Hospitals,as the core base for practical teaching,can effectively shorten the connection cycle between theoretical teaching and clinical practice by integrating AR technology,improve the clinical practice level of trainee nurses,and provide an innovative model for optimizing the path of clinical nursing talent cultivation.展开更多
Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of s...Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of surgical failure.Therefore,it is necessary to design a visualization system for closed thoracic drainage.Augmented reality(AR)technology can assist in visualizing the internal anatomical structure and determining the insertion point on the body surface.The structure of the currently used steel-needle-guided chest tube was modified by integrating it with an ultrafine diameter camera to provide real-time visualization of the puncture process.After simulation experiments,the overall registration error of the AR method was measured to be within(3.59±0.53)mm,indicating its potential for clinical application.The ultrafine diameter camera module and improved steel-needle-guided chest tube can timely reflect the position of the needle tip in the human body.A comparative experiment showed that video guidance could improve the safety of the puncture process compared to the traditional method.Finally,a qualitative evaluation of the usability of the system was conducted through a questionnaire.This system facilitates the visualization of closed thoracic drainage puncture procedure and pro-vides an implementation scheme to enhance the accuracy and safety of the operative step,which is conducive to reducing the learning curve and improving the proficiency of the doctors.展开更多
Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to...Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to visualize the optic nerve intraoperatively.To address this issue,an endoscopic image-based augmented reality surgical navigation system is developed in this study.The system aims to virtually fuse the optic nerve onto the endoscopic images,assisting surgeons in determining the optic nerve’s position and reducing surgical risks.First,a calibration algorithm based on a checkerboard grid of immobile points is proposed,building upon existing calibration methods.Additionally,to tackle accuracy issues associated with augmented reality technology,an optical navigation and visual fusion compensation algorithm is proposed to improve the intraoperative tracking accuracy.To evaluate the system’s performance,model experiments were meticulously designed and conducted.The results confirm the accuracy and stability of the proposed system,with an average tracking error of(0.99±0.46)mm.This outcome demonstrates the effectiveness of the proposed algorithm in improving the augmented reality surgical navigation system’s accuracy.Furthermore,the system successfully displays hidden optic nerves and other deep tissues,thus showcasing the promising potential for future applications in orbital and maxillofacial surgery.展开更多
The aim of this study was to assess the potential of surgical guides as a complementary tool to augmented reality(AR)in enhancing the safety and precision of pedicle screw placement in spinal surgery.Four trainers wer...The aim of this study was to assess the potential of surgical guides as a complementary tool to augmented reality(AR)in enhancing the safety and precision of pedicle screw placement in spinal surgery.Four trainers were divided into the AR navigation group using surgical guides and the free-hand group.Each group consisted of a novice and an experienced spine surgeon.A total of 80 pedicle screws were implanted.First,the AR group reconstructed the 3D model and planned the screw insertion route according to the computed tomography data of L2 lumbar vertebrae.Then,the Microsoft HoloLens™2 was used to identify the vertebral model,and the planned virtual path was superimposed on the real cone model.Next,the screw was placed according to the projected trajectory.Finally,Micron Tracker was used to measure the deviation of screws from the preoperatively planned trajectory,and pedicle screws were evaluated using the Gertzbein-Robbins scale.In the AR group,the linear deviations of the experienced doctor and the novice were(1.59±0.39)mm and(1.73±0.52)mm respectively,and the angle deviations were 2.72°±0.61°and 2.87°±0.63°respectively.In the free-hand group,the linear deviations of the experienced doctor and the novice were(2.88±0.58)mm and(5.25±0.62)mm respectively,and the angle deviations were 4.41°±1.18°and 7.15°±1.45°respectively.Both kinds of deviations between the two groups were significantly different(P<0.05).The screw accuracy rate was 95%in the AR navigation group and 77.5%in the free-hand group.The results of this study indicate that the integration of surgical guides and AR is an innovative technique that can substantially enhance the safety and precision of spinal surgery and assist inexperienced doctors in completing the surgery.展开更多
Background Augmented reality classrooms have become an interesting research topic in the field of education,but there are some limitations.Firstly,most researchers use cards to operate experiments,and a large number o...Background Augmented reality classrooms have become an interesting research topic in the field of education,but there are some limitations.Firstly,most researchers use cards to operate experiments,and a large number of cards cause difficulty and inconvenience for users.Secondly,most users conduct experiments only in the visual modal,and such single-modal interaction greatly reduces the users'real sense of interaction.In order to solve these problems,we propose the Multimodal Interaction Algorithm based on Augmented Reality(ARGEV),which is based on visual and tactile feedback in Augmented Reality.In addition,we design a Virtual and Real Fusion Interactive Tool Suite(VRFITS)with gesture recognition and intelligent equipment.Methods The ARGVE method fuses gesture,intelligent equipment,and virtual models.We use a gesture recognition model trained by a convolutional neural network to recognize the gestures in AR,and to trigger a vibration feedback after a recognizing a five finger grasp gesture.We establish a coordinate mapping relationship between real hands and the virtual model to achieve the fusion of gestures and the virtual model.Results The average accuracy rate of gesture recognition was 99.04%.We verify and apply VRFITS in the Augmented Reality Chemistry Lab(ARCL),and the overall operation load of ARCL is thus reduced by 29.42%,in comparison to traditional simulation virtual experiments.Conclusions We achieve real-time fusion of the gesture,virtual model,and intelligent equipment in ARCL.Compared with the NOBOOK virtual simulation experiment,ARCL improves the users'real sense of operation and interaction efficiency.展开更多
The impact of augmented reality(AR)technology on consumer behavior has increasingly attracted academic attention.While early research has provided valuable insights,many challenges remain.This article reviews recent s...The impact of augmented reality(AR)technology on consumer behavior has increasingly attracted academic attention.While early research has provided valuable insights,many challenges remain.This article reviews recent studies,analyzing AR’s technical features,marketing concepts,and action mechanisms from a consumer perspective.By refining existing frameworks and introducing a new model based on situation awareness theory,the paper offers a deeper exploration of AR marketing.Finally,it proposes directions for future research in this emerging field.展开更多
Background: Augmented reality(AR) technology is used to reconstruct three-dimensional(3D) images of hepatic and biliary structures from computed tomography and magnetic resonance imaging data, and to superimpose the v...Background: Augmented reality(AR) technology is used to reconstruct three-dimensional(3D) images of hepatic and biliary structures from computed tomography and magnetic resonance imaging data, and to superimpose the virtual images onto a view of the surgical field. In liver surgery, these superimposed virtual images help the surgeon to visualize intrahepatic structures and therefore, to operate precisely and to improve clinical outcomes.Data Sources: The keywords "augmented reality", "liver", "laparoscopic" and "hepatectomy" were used for searching publications in the Pub Med database. The primary source of literatures was from peer-reviewed journals up to December 2016. Additional articles were identified by manual search of references found in the key articles.Results: In general, AR technology mainly includes 3D reconstruction, display, registration as well as tracking techniques and has recently been adopted gradually for liver surgeries including laparoscopy and laparotomy with video-based AR assisted laparoscopic resection as the main technical application. By applying AR technology, blood vessels and tumor structures in the liver can be displayed during surgery,which permits precise navigation during complex surgical procedures. Liver transformation and registration errors during surgery were the main factors that limit the application of AR technology.Conclusions: With recent advances, AR technologies have the potential to improve hepatobiliary surgical procedures. However, additional clinical studies will be required to evaluate AR as a tool for reducing postoperative morbidity and mortality and for the improvement of long-term clinical outcomes. Future research is needed in the fusion of multiple imaging modalities, improving biomechanical liver modeling,and enhancing image data processing and tracking technologies to increase the accuracy of current AR methods.展开更多
To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. T...To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject＇s maxillofacial area. The three-dimensional augmented reality system （integral videography display, computed tomography, a position tracker and a computer） was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument＇s position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon＇s ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible （〈1 mm）; this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient＇s surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.展开更多
Augmented reality is widely used in aeronautics and is a developing concept within surgery. In this pilot study, we developed an application for use on Google Glass optical head-mounted display to train urology reside...Augmented reality is widely used in aeronautics and is a developing concept within surgery. In this pilot study, we developed an application for use on Google Glass optical head-mounted display to train urology residents in how to place an inflatable penile prosthesis. We use the phrase Augmented Reality Assisted Surgery to describe this novel application of augmented reality in the setting of surgery. The application demonstrates the steps of the surgical procedure of inflatable penile prosthesis placement. It also contains software that allows for detection of interest points using a camera feed from the optical head-mounted display to enable faculty to interact with residents during placement of the penile prosthesis. Urology trainees and faculty who volunteered to take part in the study were given time to experience the technology in the operative or perioperative setting and asked to complete a feedback survey. From 30 total participants using a lO-point scale, educational usefulness was rated 8.6, ease of navigation was rated 7.6, likelihood to use was rated 7.4, and distraction in operating room was rated 4.9. When stratified between trainees and faculty, trainees found the technology more educationally useful, and less distracting. Overall, 81% of the participants want this technology in their residency program, and 93% see this technology in the operating room in the future. Further development of this technology is warranted before full release, and further studies are necessary to better characterize the effectiveness of Augmented Reality Assisted Surgery in urologic surgical training.展开更多
In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking ...In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed.The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective.Then,we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption.A point cloud-based method is used to calculate the precise pose of the camera in real time.In order to solve the problem that the traditional iterative closest point(ICP)algorithm cannot meet the real-time requirements of the system,Kdtree(k-dimensional tree)is used under the graphics processing unit(GPU)to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration.At the same time,the random sample consensus(RANSAC)algorithm is used to remove the error point pairs to improve the accuracy of the algorithm.The results show that the proposed tracking registration method has good real-time performance and robustness.展开更多
Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network ...Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network is prone to getting into local extrema and convergence is slow, genetic algorithm is employed to optimize the initial weights and threshold of neural network. This paper discusses how to set the crucial parameters in the algorithm. Experimental results show that the method ensures that the neural network achieves global convergence quickly and correctly. Tracking precision of AR system is improved after the tracker is rectified, and the third dimension of AR system is enhanced.展开更多
We aim to develop a novel visualization tool for percutaneous renal puncture training based on augmented reality(AR)and compare the needle placement performance of this AR system with ultrasound-guidedfreehand naviga...We aim to develop a novel visualization tool for percutaneous renal puncture training based on augmented reality(AR)and compare the needle placement performance of this AR system with ultrasound-guidedfreehand navigation in phantoms.A head-mounted display-based AR navigation system was developed usingthe Unity3D software and Visual Studio to enable the overlay of the preoperative needle path and the complexanatomical structures onto a phantom in real time.The spatial location of the stationary phantom and the percutaneous instrument motion were traced by a Qualisys motion capture system.To evaluate the tracking accuracy,15 participants(7 males and 8 females)performed a single needle insertion using AR navigation(the number ofpunctures n=75)and ultrasound-guided freehand navigation(n=75).The needle placement error was measuredas the Euclidean distance between the actual needle tip and the virtual target by MicronTracker.All participantsdemonstrated a superior needle insertion efficiency when using the AR-assisted puncture method compared withthe ultrasound-guided freehand method.The needle insertion error of the ultrasound-guided method showed anincreased error compared with the AR method(5.54 mm±2.59 mm,4.34 mm±2.10 mm,respectively,p<0.05).The ultrasound-guided needle placements showed an increased time compared with the AR method(19.08 s±3.59 s,15.14 s±2.72 s,respectively,p<0.0001).Our AR training system facilitates the needle placement performance and solves hand-eye coordination problems.The system has the potential to increase efficiency andeffectiveness of percutaneous renal puncture training.展开更多
Vision 2030 requires a new generation of people with a wide variety of abilities,talents,and skills.The adoption of augmented reality(AR)and virtual reality is one possible way to align education with Vision 2030.Imme...Vision 2030 requires a new generation of people with a wide variety of abilities,talents,and skills.The adoption of augmented reality(AR)and virtual reality is one possible way to align education with Vision 2030.Immersive technologies like AR are rapidly becoming powerful and versatile enough to be adopted in education to achieve this goal.Technologies such as AR could be beneficial tools to enhance maintainable growth in education.We reviewed the most recent studies in augmented reality to check its appropriateness in aligning with the educational goals of Vision 2030.First,the various definitions,terminologies,and technologies of AR are described briefly.Then,the specific characteristics and benefits of AR systems are determined.There may be a significance of the pedagogical method used by adapting the AR scheme and the consistency of the equipment and learning experiences.Therefore,three kinds of instructional methods that stress roles,location,and tasks were evaluated.The kind of learning that is offered by the distinct kinds of AR approaches is elaborated upon.The technological,pedagogical,learning problems experienced with AR are described.The potential solutions for a few of the issues experienced and the topics for subsequent research are presented in this article.展开更多
BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolvi...BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.展开更多
Product assembly simulation is considered as one of the key technologies in the process of complex product design and manufacturing.Virtual assembly realizes the assembly process design,verification,and optimization o...Product assembly simulation is considered as one of the key technologies in the process of complex product design and manufacturing.Virtual assembly realizes the assembly process design,verification,and optimization of complex products in the virtual environment,which plays an active and effective role in improving the assembly quality and efficiency of complex products.In recent years,augmented reality(AR)and digital twin(DT)technology have brought new opportunities and challenges to the digital assembly of complex products owing to their characteristics of virtual reality fusion and interactive control.This paper expounds the concept and connotation of AR,enumerates a typical AR assembly system structure,analyzes the key technologies and applications of AR in digital assembly,and notes that DT technology is the future development trend of intelligent assembly research.展开更多
Augmented reality(AR)displays are attracting significant attention and efforts.In this paper,we review the adopted device configurations of see-through displays,summarize the current development status and highlight f...Augmented reality(AR)displays are attracting significant attention and efforts.In this paper,we review the adopted device configurations of see-through displays,summarize the current development status and highlight future challenges in micro-displays.A brief introduction to optical gratings is presented to help understand the challenging design of grating-based waveguide for AR displays.Finally,we discuss the most recent progress in diffraction grating and its implications.展开更多
文摘Augmented reality(AR)is an emerging dynamic technology that effectively supports education across different levels.The increased use of mobile devices has an even greater impact.As the demand for AR applications in education continues to increase,educators actively seek innovative and immersive methods to engage students in learning.However,exploring these possibilities also entails identifying and overcoming existing barriers to optimal educational integration.Concurrently,this surge in demand has prompted the identification of specific barriers,one of which is three-dimensional(3D)modeling.Creating 3D objects for augmented reality education applications can be challenging and time-consuming for the educators.To address this,we have developed a pipeline that creates realistic 3D objects from the two-dimensional(2D)photograph.Applications for augmented and virtual reality can then utilize these created 3D objects.We evaluated the proposed pipeline based on the usability of the 3D object and performance metrics.Quantitatively,with 117 respondents,the co-creation team was surveyed with openended questions to evaluate the precision of the 3D object created by the proposed photogrammetry pipeline.We analyzed the survey data using descriptive-analytical methods and found that the proposed pipeline produces 3D models that are positively accurate when compared to real-world objects,with an average mean score above 8.This study adds new knowledge in creating 3D objects for augmented reality applications by using the photogrammetry technique;finally,it discusses potential problems and future research directions for 3D objects in the education sector.
基金the University of Shanghai for Science and Technology’s Medical Engineering Interdisciplinary Project(No.10-22-308-520)the Ministry of Education’s First Batch of Industry-Education Cooperation Collaborative Education Projects(No.202101042008)+1 种基金the Fundamental Research Funds for the Central Universities(No.YG2019QNA34)the Shanghai Municipal Health Commission for Youth Clinical Research Project(No.20194Y0134)。
文摘The purpose of this study is to establish a multivariate nonlinear regression mathematical model to predict the displacement of tumor during brain tumor resection surgery.And the study will be integrated with augmented reality technology to achieve three-dimensional visualization,thereby enhancing the complete resection rate of tumor and the success rate of surgery.Based on the preoperative MRI data of the patients,a 3D virtual model is reconstructed and 3D printed.A brain biomimetic model is created using gel injection molding.By considering cerebrospinal fluid loss and tumor cyst fluid loss as independent variables,the highest point displacement in the vertical bone window direction is determined as the dependent variable after positioning the patient for surgery.An orthogonal experiment is conducted on the biomimetic model to establish a predictive model,and this model is incorporated into the augmented reality navigation system.To validate the predictive model,five participants wore HoloLens2 devices,overlaying the patient’s 3D virtual model onto the physical head model.Subsequently,the spatial coordinates of the tumor’s highest point after displacement were measured on both the physical and virtual models(actual coordinates and predicted coordinates,respectively).The difference between these coordinates represents the model’s prediction error.The results indicate that the measured and predicted errors for the displacement of the tumor’s highest point on the X and Y axes range from−0.6787 mm to 0.2957 mm and−0.4314 mm to 0.2253 mm,respectively.The relative errors for each experimental group are within 10%,demonstrating a good fit of the model.This method of establishing a regression model represents a preliminary attempt to predict brain tumor displacement in specific situations.It also provides a new approach for surgeons.By combining augmented reality visualization,it addresses the need for predicting tumor displacement and precisely locating brain anatomical structures in a simple and cost-effective manner.
基金Supported by The Hunan Provincial Natural Science Foundation of China,No.2023JJ30773,No.2025JJ60480,and No.2025JJ60552The Scientific Research Program of The Hunan Provincial Health Commission,No.202204072544+4 种基金The Science and Technology Innovation Program of Hunan Province,No.2024RC3053The CBT ECR/MCR Scheme,No.324910-0028/07National Natural Science Foundation of China,No.32300652The Scientific Research Program of Hunan Provincial Health Commission,No.W20243023The Scientific Research Launch Project for New Employees of The Second Xiangya Hospital of Central South University.
文摘Augmented reality(AR)is a technology that superimposes digital information onto real-world objects via head-mounted display devices to improve surgical finesse through visually enhanced medical information.With the rapid development of digital technology,AR has been increasingly adopted in orthopedic surgeries across the globe,especially in total knee arthroplasty procedures which demand high precision.By overlaying digital information onto the surgeon's field of view,AR systems enhance precision,improve alignment accuracy,and reduce the risk of complications associated with malalignment.Some concerns have been raised despite accuracy,including the learning curve,long-term outcomes,and technical limitations.Furthermore,it is essential for health practitioners to gain trust in the utilisation of AR.
文摘With the iteration and upgrading of medical technology and the continuous growth of public health demands,the quality of nursing services has become a core indicator for measuring the effectiveness of the medical system.The clinical practice ability of nursing staff is directly related to the safety of patient diagnosis and treatment and the rehabilitation process.However,the current clinical nursing talent training model is facing bottlenecks such as limited practical scenarios and fragmented case cognition.This study focuses on the teaching application of augmented reality(AR)technology in hospital Settings and systematically reviews the research progress on the improvement of clinical practice ability of trainee nurses based on the AR immersive teaching model.By constructing a clinical teaching scenario that integrates virtual and real,AR technology can dynamically simulate complex case handling processes and enhance nursing students’three-dimensional cognition of condition assessment,operation norms,and emergency plans.Hospitals,as the core base for practical teaching,can effectively shorten the connection cycle between theoretical teaching and clinical practice by integrating AR technology,improve the clinical practice level of trainee nurses,and provide an innovative model for optimizing the path of clinical nursing talent cultivation.
基金the Shanghai Municipal Education Commission-Gaofeng Clinical Medicine Grant(No.20172005)。
文摘Closed thoracic drainage can be performed using a steel-needle-guided chest tube to treat pleural effusion or pneumothorax in clinics.However,the puncture procedure during surgery is invisible,increasing the risk of surgical failure.Therefore,it is necessary to design a visualization system for closed thoracic drainage.Augmented reality(AR)technology can assist in visualizing the internal anatomical structure and determining the insertion point on the body surface.The structure of the currently used steel-needle-guided chest tube was modified by integrating it with an ultrafine diameter camera to provide real-time visualization of the puncture process.After simulation experiments,the overall registration error of the AR method was measured to be within(3.59±0.53)mm,indicating its potential for clinical application.The ultrafine diameter camera module and improved steel-needle-guided chest tube can timely reflect the position of the needle tip in the human body.A comparative experiment showed that video guidance could improve the safety of the puncture process compared to the traditional method.Finally,a qualitative evaluation of the usability of the system was conducted through a questionnaire.This system facilitates the visualization of closed thoracic drainage puncture procedure and pro-vides an implementation scheme to enhance the accuracy and safety of the operative step,which is conducive to reducing the learning curve and improving the proficiency of the doctors.
基金the National Natural Science Foundation of China(Nos.82330063 and M-0019)the Interdisciplinary Program of Shanghai Jiao Tong University(Nos.YG2022QN056,YG2023ZD19,and YG2023ZD15)+2 种基金the Cross Disciplinary Research Fund of Shanghai Ninth People’s Hospital,Shanghai Jiao Tong University School of Medicine(No.JYJC202115)the Translation Clinical R&D Project of Medical Robot of Shanghai Ninth People’s Hospital,Shanghai Jiao Tong University School of Medicine(No.IMR-NPH202002)the Shanghai Key Clinical Specialty,Shanghai Eye Disease Research Center(No.2022ZZ01003)。
文摘Endoscopic transnasal optic nerve decompression surgery plays a crucial role in minimal invasive treatment of complex traumatic optic neuropathy.However,a major challenge faced during the procedure is the inability to visualize the optic nerve intraoperatively.To address this issue,an endoscopic image-based augmented reality surgical navigation system is developed in this study.The system aims to virtually fuse the optic nerve onto the endoscopic images,assisting surgeons in determining the optic nerve’s position and reducing surgical risks.First,a calibration algorithm based on a checkerboard grid of immobile points is proposed,building upon existing calibration methods.Additionally,to tackle accuracy issues associated with augmented reality technology,an optical navigation and visual fusion compensation algorithm is proposed to improve the intraoperative tracking accuracy.To evaluate the system’s performance,model experiments were meticulously designed and conducted.The results confirm the accuracy and stability of the proposed system,with an average tracking error of(0.99±0.46)mm.This outcome demonstrates the effectiveness of the proposed algorithm in improving the augmented reality surgical navigation system’s accuracy.Furthermore,the system successfully displays hidden optic nerves and other deep tissues,thus showcasing the promising potential for future applications in orbital and maxillofacial surgery.
基金the National Natural Science Foundation of China(No.11502146)the 1 Batch of 2021 MOE of PRC Industry University Collaborative Education Program(No.202101042008)。
文摘The aim of this study was to assess the potential of surgical guides as a complementary tool to augmented reality(AR)in enhancing the safety and precision of pedicle screw placement in spinal surgery.Four trainers were divided into the AR navigation group using surgical guides and the free-hand group.Each group consisted of a novice and an experienced spine surgeon.A total of 80 pedicle screws were implanted.First,the AR group reconstructed the 3D model and planned the screw insertion route according to the computed tomography data of L2 lumbar vertebrae.Then,the Microsoft HoloLens™2 was used to identify the vertebral model,and the planned virtual path was superimposed on the real cone model.Next,the screw was placed according to the projected trajectory.Finally,Micron Tracker was used to measure the deviation of screws from the preoperatively planned trajectory,and pedicle screws were evaluated using the Gertzbein-Robbins scale.In the AR group,the linear deviations of the experienced doctor and the novice were(1.59±0.39)mm and(1.73±0.52)mm respectively,and the angle deviations were 2.72°±0.61°and 2.87°±0.63°respectively.In the free-hand group,the linear deviations of the experienced doctor and the novice were(2.88±0.58)mm and(5.25±0.62)mm respectively,and the angle deviations were 4.41°±1.18°and 7.15°±1.45°respectively.Both kinds of deviations between the two groups were significantly different(P<0.05).The screw accuracy rate was 95%in the AR navigation group and 77.5%in the free-hand group.The results of this study indicate that the integration of surgical guides and AR is an innovative technique that can substantially enhance the safety and precision of spinal surgery and assist inexperienced doctors in completing the surgery.
基金the National Key R&D Program of China(2018YFB1004901)the Independent Innovation Team Project of Jinan City(2019GXRC013).
文摘Background Augmented reality classrooms have become an interesting research topic in the field of education,but there are some limitations.Firstly,most researchers use cards to operate experiments,and a large number of cards cause difficulty and inconvenience for users.Secondly,most users conduct experiments only in the visual modal,and such single-modal interaction greatly reduces the users'real sense of interaction.In order to solve these problems,we propose the Multimodal Interaction Algorithm based on Augmented Reality(ARGEV),which is based on visual and tactile feedback in Augmented Reality.In addition,we design a Virtual and Real Fusion Interactive Tool Suite(VRFITS)with gesture recognition and intelligent equipment.Methods The ARGVE method fuses gesture,intelligent equipment,and virtual models.We use a gesture recognition model trained by a convolutional neural network to recognize the gestures in AR,and to trigger a vibration feedback after a recognizing a five finger grasp gesture.We establish a coordinate mapping relationship between real hands and the virtual model to achieve the fusion of gestures and the virtual model.Results The average accuracy rate of gesture recognition was 99.04%.We verify and apply VRFITS in the Augmented Reality Chemistry Lab(ARCL),and the overall operation load of ARCL is thus reduced by 29.42%,in comparison to traditional simulation virtual experiments.Conclusions We achieve real-time fusion of the gesture,virtual model,and intelligent equipment in ARCL.Compared with the NOBOOK virtual simulation experiment,ARCL improves the users'real sense of operation and interaction efficiency.
基金Guizhou University of Finance and Economics 2024 Student Self-Funded Research Project Funding(Project no.2024ZXSY001)。
文摘The impact of augmented reality(AR)technology on consumer behavior has increasingly attracted academic attention.While early research has provided valuable insights,many challenges remain.This article reviews recent studies,analyzing AR’s technical features,marketing concepts,and action mechanisms from a consumer perspective.By refining existing frameworks and introducing a new model based on situation awareness theory,the paper offers a deeper exploration of AR marketing.Finally,it proposes directions for future research in this emerging field.
基金supported by grants from the Mission Plan Program of Beijing Municipal Administration of Hospitals(SML20152201)Beijing Municipal Administration of Hospitals Clinical Medicine Development of Special Funding(ZYLX201712)+1 种基金the National Natural Science Foundation of China(81427803)Beijing Tsinghua Changgung Hospital Fund(12015C1039)
文摘Background: Augmented reality(AR) technology is used to reconstruct three-dimensional(3D) images of hepatic and biliary structures from computed tomography and magnetic resonance imaging data, and to superimpose the virtual images onto a view of the surgical field. In liver surgery, these superimposed virtual images help the surgeon to visualize intrahepatic structures and therefore, to operate precisely and to improve clinical outcomes.Data Sources: The keywords "augmented reality", "liver", "laparoscopic" and "hepatectomy" were used for searching publications in the Pub Med database. The primary source of literatures was from peer-reviewed journals up to December 2016. Additional articles were identified by manual search of references found in the key articles.Results: In general, AR technology mainly includes 3D reconstruction, display, registration as well as tracking techniques and has recently been adopted gradually for liver surgeries including laparoscopy and laparotomy with video-based AR assisted laparoscopic resection as the main technical application. By applying AR technology, blood vessels and tumor structures in the liver can be displayed during surgery,which permits precise navigation during complex surgical procedures. Liver transformation and registration errors during surgery were the main factors that limit the application of AR technology.Conclusions: With recent advances, AR technologies have the potential to improve hepatobiliary surgical procedures. However, additional clinical studies will be required to evaluate AR as a tool for reducing postoperative morbidity and mortality and for the improvement of long-term clinical outcomes. Future research is needed in the fusion of multiple imaging modalities, improving biomechanical liver modeling,and enhancing image data processing and tracking technologies to increase the accuracy of current AR methods.
基金supported by a Grant-in-Aid for Scientific Research (22659366) from the Japan Society for the Promotion of Science
文摘To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject＇s maxillofacial area. The three-dimensional augmented reality system （integral videography display, computed tomography, a position tracker and a computer） was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument＇s position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon＇s ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible （〈1 mm）; this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient＇s surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.
文摘Augmented reality is widely used in aeronautics and is a developing concept within surgery. In this pilot study, we developed an application for use on Google Glass optical head-mounted display to train urology residents in how to place an inflatable penile prosthesis. We use the phrase Augmented Reality Assisted Surgery to describe this novel application of augmented reality in the setting of surgery. The application demonstrates the steps of the surgical procedure of inflatable penile prosthesis placement. It also contains software that allows for detection of interest points using a camera feed from the optical head-mounted display to enable faculty to interact with residents during placement of the penile prosthesis. Urology trainees and faculty who volunteered to take part in the study were given time to experience the technology in the operative or perioperative setting and asked to complete a feedback survey. From 30 total participants using a lO-point scale, educational usefulness was rated 8.6, ease of navigation was rated 7.6, likelihood to use was rated 7.4, and distraction in operating room was rated 4.9. When stratified between trainees and faculty, trainees found the technology more educationally useful, and less distracting. Overall, 81% of the participants want this technology in their residency program, and 93% see this technology in the operating room in the future. Further development of this technology is warranted before full release, and further studies are necessary to better characterize the effectiveness of Augmented Reality Assisted Surgery in urologic surgical training.
基金This work was supported by National Natural Science Foundation of China(No.61125101).
文摘In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality,a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed.The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective.Then,we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption.A point cloud-based method is used to calculate the precise pose of the camera in real time.In order to solve the problem that the traditional iterative closest point(ICP)algorithm cannot meet the real-time requirements of the system,Kdtree(k-dimensional tree)is used under the graphics processing unit(GPU)to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration.At the same time,the random sample consensus(RANSAC)algorithm is used to remove the error point pairs to improve the accuracy of the algorithm.The results show that the proposed tracking registration method has good real-time performance and robustness.
基金Project supported by Science Foundation of Shanghai Municipal Commission of Science and Technology (Grant No .025115008)
文摘Nonlinear errors always exist in data obtained from tracker in augmented reality （AR）, which badly influence the effect of AR. This paper proposes to rectify the errors using BP neural network. As BP neural network is prone to getting into local extrema and convergence is slow, genetic algorithm is employed to optimize the initial weights and threshold of neural network. This paper discusses how to set the crucial parameters in the algorithm. Experimental results show that the method ensures that the neural network achieves global convergence quickly and correctly. Tracking precision of AR system is improved after the tracker is rectified, and the third dimension of AR system is enhanced.
基金supported by Tianjin Sci-tech Planning Projects (14RCGFGX00846)Natural Science Foundation of Hebei Province (F2015202239)+1 种基金Tianjin Sci-tech Planning Projects (15ZCZDNC00130)Science and Technology Research Project of Hebei Province (Z2015044)
基金the National Natural Science Foundation of China(No.11502146)。
文摘We aim to develop a novel visualization tool for percutaneous renal puncture training based on augmented reality(AR)and compare the needle placement performance of this AR system with ultrasound-guidedfreehand navigation in phantoms.A head-mounted display-based AR navigation system was developed usingthe Unity3D software and Visual Studio to enable the overlay of the preoperative needle path and the complexanatomical structures onto a phantom in real time.The spatial location of the stationary phantom and the percutaneous instrument motion were traced by a Qualisys motion capture system.To evaluate the tracking accuracy,15 participants(7 males and 8 females)performed a single needle insertion using AR navigation(the number ofpunctures n=75)and ultrasound-guided freehand navigation(n=75).The needle placement error was measuredas the Euclidean distance between the actual needle tip and the virtual target by MicronTracker.All participantsdemonstrated a superior needle insertion efficiency when using the AR-assisted puncture method compared withthe ultrasound-guided freehand method.The needle insertion error of the ultrasound-guided method showed anincreased error compared with the AR method(5.54 mm±2.59 mm,4.34 mm±2.10 mm,respectively,p<0.05).The ultrasound-guided needle placements showed an increased time compared with the AR method(19.08 s±3.59 s,15.14 s±2.72 s,respectively,p<0.0001).Our AR training system facilitates the needle placement performance and solves hand-eye coordination problems.The system has the potential to increase efficiency andeffectiveness of percutaneous renal puncture training.
文摘Vision 2030 requires a new generation of people with a wide variety of abilities,talents,and skills.The adoption of augmented reality(AR)and virtual reality is one possible way to align education with Vision 2030.Immersive technologies like AR are rapidly becoming powerful and versatile enough to be adopted in education to achieve this goal.Technologies such as AR could be beneficial tools to enhance maintainable growth in education.We reviewed the most recent studies in augmented reality to check its appropriateness in aligning with the educational goals of Vision 2030.First,the various definitions,terminologies,and technologies of AR are described briefly.Then,the specific characteristics and benefits of AR systems are determined.There may be a significance of the pedagogical method used by adapting the AR scheme and the consistency of the equipment and learning experiences.Therefore,three kinds of instructional methods that stress roles,location,and tasks were evaluated.The kind of learning that is offered by the distinct kinds of AR approaches is elaborated upon.The technological,pedagogical,learning problems experienced with AR are described.The potential solutions for a few of the issues experienced and the topics for subsequent research are presented in this article.
文摘BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years,as they enhance precision compared to conventional hardware.The expansion of computer assistance is evolving with the employment of augmented reality.Yet,the accuracy of augmented reality navigation systems has not been determined.AIM To examine the accuracy of component alignment and restoration of the affected limb’s mechanical axis in primary total knee arthroplasty(TKA),utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.METHODS From May 2021 to December 2021,30 patients,25 women and five men,under-went a primary unilateral TKA.Revision cases were excluded.A preoperative radiographic procedure was performed to evaluate the limb’s axial alignment.All patients were operated on by the same team,without a tourniquet,utilizing three distinct prostheses with the assistance of the Knee+™augmented reality navigation system in every operation.Postoperatively,the same radiographic exam protocol was executed to evaluate the implants’position,orientation and coronal plane alignment.We recorded measurements in 3 stages regarding femoral varus and flexion,tibial varus and posterior slope.Firstly,the expected values from the Augmented Reality system were documented.Then we calculated the same values after each cut and finally,the same measurements were recorded radiolo-gically after the operations.Concerning statistical analysis,Lin’s concordance correlation coefficient was estimated,while Wilcoxon Signed Rank Test was performed when needed.RESULTS A statistically significant difference was observed regarding mean expected values and radiographic mea-surements for femoral flexion measurements only(Z score=2.67,P value=0.01).Nonetheless,this difference was statistically significantly lower than 1 degree(Z score=-4.21,P value<0.01).In terms of discrepancies in the calculations of expected values and controlled measurements,a statistically significant difference between tibial varus values was detected(Z score=-2.33,P value=0.02),which was also statistically significantly lower than 1 degree(Z score=-4.99,P value<0.01).CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized.Augmented reality navigation systems can bolster orthopaedic surgeons’accuracy in achieving precise axial alignment.However,further research is required to further evaluate their efficacy and potential.
基金the National Natural Science Foundation of China(51875517,51490663 and 51821093)and Key Research and Development Program of Zhejiang Province(2017C01045).
文摘Product assembly simulation is considered as one of the key technologies in the process of complex product design and manufacturing.Virtual assembly realizes the assembly process design,verification,and optimization of complex products in the virtual environment,which plays an active and effective role in improving the assembly quality and efficiency of complex products.In recent years,augmented reality(AR)and digital twin(DT)technology have brought new opportunities and challenges to the digital assembly of complex products owing to their characteristics of virtual reality fusion and interactive control.This paper expounds the concept and connotation of AR,enumerates a typical AR assembly system structure,analyzes the key technologies and applications of AR in digital assembly,and notes that DT technology is the future development trend of intelligent assembly research.
基金Air Force Office of Scientific Research(FA9550-14-1-0279)Goertek Electronics.
文摘Augmented reality(AR)displays are attracting significant attention and efforts.In this paper,we review the adopted device configurations of see-through displays,summarize the current development status and highlight future challenges in micro-displays.A brief introduction to optical gratings is presented to help understand the challenging design of grating-based waveguide for AR displays.Finally,we discuss the most recent progress in diffraction grating and its implications.
