In response to the construction needs of “Real 3D China”, the system structure, functional framework, application direction and product form of block level augmented reality three-dimensional map is designed. Those ...In response to the construction needs of “Real 3D China”, the system structure, functional framework, application direction and product form of block level augmented reality three-dimensional map is designed. Those provide references and ideas for the later large-scale production of augmented reality three-dimensional map. The augmented reality three-dimensional map is produced based on skyline software. Including the map browsing, measurement and analysis and so on, the basic function of three-dimensional map is realized. The special functional module including housing management, pipeline management and so on is developed combining the need of residential quarters development, that expands the application fields of augmented reality three-dimensional map. Those lay the groundwork for the application of augmented reality three-dimensional map. .展开更多
To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. T...To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject＇s maxillofacial area. The three-dimensional augmented reality system （integral videography display, computed tomography, a position tracker and a computer） was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument＇s position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon＇s ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible （〈1 mm）; this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient＇s surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.展开更多
Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal i...Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal interventions for cancer treatment.Methods:A specialized software transforms 2D medical images into precise 3D digital models, facilitating improved anatomical understanding and surgical planning. Patient-specific 3D printed anatomical models are utilized for preoperative planning, intraoperative guidance, and surgical education. AR technology enables the overlay of digital perceptions onto real-world surgical environments.Results:Patient-specific 3D printed anatomical models have multiple applications, such as preoperative planning, intraoperative guidance, trainee education, and patient counseling. Virtual reality involves substituting the real world with a computer-generated 3D environment, while AR overlays digitally created perceptions onto the existing reality. The advances in 3D modeling technology have sparked considerable interest in their application to partial nephrectomy in the realm of renal cancer. 3D printing, also known as additive manufacturing, constructs 3D objects based on computer-aided design or digital 3D models. Utilizing 3D-printed preoperative renal models provides benefits for surgical planning, offering a more reliable assessment of the tumor's relationship with vital anatomical structures and enabling better preparation for procedures. AR technology allows surgeons to visualize patient-specific renal anatomical structures and their spatial relationships with surrounding organs by projecting CT/MRI images onto a live laparoscopic video. Incorporating patient-specific 3D digital models into healthcare enhances best practice, resulting in improved patient care, increased patient satisfaction, and cost saving for the healthcare system.展开更多
This research studies the process of 3D reconstruction and dynamic concision based on 2D medical digital images using virtual reality modelling language (VRML) and JavaScript language, with a focus on how to realize t...This research studies the process of 3D reconstruction and dynamic concision based on 2D medical digital images using virtual reality modelling language (VRML) and JavaScript language, with a focus on how to realize the dynamic concision of 3D medical model with script node and sensor node in VRML. The 3D reconstruction and concision of body internal organs can be built with such high quality that they are better than those obtained from the traditional methods. With the function of dynamic concision, the VRML browser can offer better windows for man-computer interaction in real-time environment than ever before. 3D reconstruction and dynamic concision with VRML can be used to meet the requirement for the medical observation of 3D reconstruction and have a promising prospect in the fields of medical imaging.展开更多
Although VSLAM/VISLAM has achieved great success,it is still difficult to quantitatively evaluate the localization results of different kinds of SLAM systems from the aspect of augmented reality due to the lack of an ...Although VSLAM/VISLAM has achieved great success,it is still difficult to quantitatively evaluate the localization results of different kinds of SLAM systems from the aspect of augmented reality due to the lack of an appropriate benchmark.For AR applications in practice,a variety of challenging situations(e.g.,fast motion,strong rotation,serious motion blur,dynamic interference)may be easily encountered since a home user may not carefully move the AR device,and the real environment may be quite complex.In addition,the frequency of camera lost should be minimized and the recovery from the failure status should be fast and accurate for good AR experience.Existing SLAM datasets/benchmarks generally only provide the evaluation of pose accuracy and their camera motions are somehow simple and do not fit well the common cases in the mobile AR applications.With the above motivation,we build a new visual-inertial dataset as well as a series of evaluation criteria for AR.We also review the existing monocular VSLAM/VISLAM approaches with detailed analyses and comparisons.Especially,we select 8 representative monocular VSLAM/VISLAM approaches/systems and quantitatively evaluate them on our benchmark.Our dataset,sample code and corresponding evaluation tools are available at the benchmark website http://www.zjucvg.net/eval-vislam/.展开更多
Realism rendering methods of outdoor augmented reality(AR)is an interesting topic.Realism items in outdoor AR need advanced impacts like shadows,sunshine,and relations between unreal items.A few realistic rendering ap...Realism rendering methods of outdoor augmented reality(AR)is an interesting topic.Realism items in outdoor AR need advanced impacts like shadows,sunshine,and relations between unreal items.A few realistic rendering approaches were built to overcome this issue.Several of these approaches are not dealt with real-time rendering.However,the issue remains an active research topic,especially in outdoor rendering.This paper introduces a new approach to accomplish reality real-time outdoor rendering by considering the relation between items in AR regarding shadows in any place during daylight.The proposed method includes three principal stages that cover various outdoor AR rendering challenges.First,real shadow recognition was generated considering the sun’s location and the intensity of the shadow.The second step involves real shadow protection.Finally,we introduced a shadow production algorithm technique and shades through its impacts on unreal items in the AR.The selected approach’s target is providing a fast shadow recognition technique without affecting the system’s accuracy.It achieved an average accuracy of 95.1%and an area under the curve(AUC)of 92.5%.The outputs demonstrated that the proposed approach had enhanced the reality of outside AR rendering.The results of the proposed method outperformed other state-of-the-art rendering shadow techniques’outcomes.展开更多
Implementing augmented reality(AR)and three-dimensional(3D)imaging within routine surgical practice is challenging.This requires considerable time,investment,and perseverance.Recently,the application of such technolog...Implementing augmented reality(AR)and three-dimensional(3D)imaging within routine surgical practice is challenging.This requires considerable time,investment,and perseverance.Recently,the application of such technologies has been scrutinized within the context of urology across multiple settings,including preoperative planning,intraoperative diagnostics,training,and counseling.^(l)Specific attention has been paid to standard anterior robot-assisted radical prostatectomy(RARP),where AR has demonstrated promising results.展开更多
Objective:To highlight the role of hyper accuracy three-dimensional(3D)reconstruction in facilitating surgical planning and guiding selective clamping during robot-assisted partial nephrectomy(RAPN).Methods:A transper...Objective:To highlight the role of hyper accuracy three-dimensional(3D)reconstruction in facilitating surgical planning and guiding selective clamping during robot-assisted partial nephrectomy(RAPN).Methods:A transperitoneal RAPN was performed in a 62-year-old male patient presenting with a 4 cm right anterior interpolar renal mass(R.E.N.A.L nephrometry score 7A).An abnormal vasculature was observed,with a single renal vein and two right renal arteries originating superiorly to the vein and anterior,when dividing in their segmental branches.According to the hyper accuracy 3D(HA3D^(®))rainbow model(MEDICS Srl,Turin,Italy),one branch belonging to one of the segmental arteries was feeding the tumor.This allowed for an accurate prediction of the area vascularized by each arterial branch.The 3D model was included in the intraoperative console view during the whole procedure,using the TilePro feature.A step-by-step explanation of the procedure is provided in the video attached to the present article.Results:The operative time was 90 min with a warm ischemia time on selective clamping of 13 min.Estimated blood loss was 180 mL.No intraoperative complication was encountered and no drain was placed at the end of the procedure.The patient was discharged on postoperative Day 2,without any early postoperative complications.The final pathology report showed a pathological tumor stage 1 clear cell renal cell carcinoma with negative surgical margins.Conclusion:The present study and the attached video illustrate the value of 3D rainbow model during the planning and execution of a RAPN with selective clamping.It shows how the surgeon can rely on this model to be more efficient by avoiding unnecessary surgical steps,and to safely adopt a“selective”clamping strategy that can translate in minimal functional impact.展开更多
The landscape of hepatobiliary surgical education has undergone a significanttransformation with the integration of advanced technologies such as threedimensionalmodeling,virtual reality,augmented reality,and artifici...The landscape of hepatobiliary surgical education has undergone a significanttransformation with the integration of advanced technologies such as threedimensionalmodeling,virtual reality,augmented reality,and artificial intelligence.This review synthesizes recent advancements in surgical education,examiningthe role of these technologies in improving anatomical understanding,surgicalskill acquisition,and overall trainee engagement.Evidence from randomizedcontrolled trials,systematic reviews,and cohort studies shows that immersivetraining tools,including virtual reality,augmented reality,and haptic feedback,outperform traditional apprenticeship methods in fostering cognitive and psychomotorskills.Artificial intelligence applications provide real-time feedback,furtherenhancing learning efficiency.However,these technologies should complement,rather than replace,traditional hands-on training.Some challenges remain to beaddressed,such as high costs,infrastructure requirements,and limited long-termvalidation of these technologies.The review concludes that while these innovationsoffer promising educational benefits,further research is needed to standardizetheir application and evaluate their long-term impact on surgical outcomes.展开更多
Unlike earlier innovations that focused on manipulation and task precision,the latest surgical advancements emphasize data visualization and the acceleration of surgeons’decision-making.Before entering the operating ...Unlike earlier innovations that focused on manipulation and task precision,the latest surgical advancements emphasize data visualization and the acceleration of surgeons’decision-making.Before entering the operating room,digital tools help surgeons develop patient-specific approaches,outline procedural steps,and prepare strategies for managing potential crises.During the operation,relevant data appear at the right moment to assist decision-making and guide performance in formats tailored to the surgeon’s physical and cognitive state,all without causing distractions.In the future operating room,performance data from surgeons will be gathered,integrated,and analyzed to interpret surgeons’behavior and confidence.Advanced techniques,including artificial intelligence,can be employed to automatically detect moments of performance difficulty or cognitive overload.As a result,digital surgery provides timely,context-specific assistance to surgeons in high-stakes situations.Key components of digital surgery are highlighted in this narrative review.展开更多
文摘In response to the construction needs of “Real 3D China”, the system structure, functional framework, application direction and product form of block level augmented reality three-dimensional map is designed. Those provide references and ideas for the later large-scale production of augmented reality three-dimensional map. The augmented reality three-dimensional map is produced based on skyline software. Including the map browsing, measurement and analysis and so on, the basic function of three-dimensional map is realized. The special functional module including housing management, pipeline management and so on is developed combining the need of residential quarters development, that expands the application fields of augmented reality three-dimensional map. Those lay the groundwork for the application of augmented reality three-dimensional map. .
基金supported by a Grant-in-Aid for Scientific Research (22659366) from the Japan Society for the Promotion of Science
文摘To evaluate the feasibility and accuracy of a three-dimensional augmented reality system incorporating integral videography for imaging oral and maxillofacial regions, based on preoperative computed tomography data. Three-dimensional surface models of the jawbones, based on the computed tomography data, were used to create the integral videography images of a subject＇s maxillofacial area. The three-dimensional augmented reality system （integral videography display, computed tomography, a position tracker and a computer） was used to generate a three-dimensional overlay that was projected on the surgical site via a half-silvered mirror. Thereafter, a feasibility study was performed on a volunteer. The accuracy of this system was verified on a solid model while simulating bone resection. Positional registration was attained by identifying and tracking the patient/surgical instrument＇s position. Thus, integral videography images of jawbones, teeth and the surgical tool were superimposed in the correct position. Stereoscopic images viewed from various angles were accurately displayed. Change in the viewing angle did not negatively affect the surgeon＇s ability to simultaneously observe the three-dimensional images and the patient, without special glasses. The difference in three-dimensional position of each measuring point on the solid model and augmented reality navigation was almost negligible （〈1 mm）; this indicates that the system was highly accurate. This augmented reality system was highly accurate and effective for surgical navigation and for overlaying a three-dimensional computed tomography image on a patient＇s surgical area, enabling the surgeon to understand the positional relationship between the preoperative image and the actual surgical site, with the naked eye.
文摘Objective:This study aimed to explore the applications of three-dimensional (3D) technology, including virtual reality, augmented reality (AR), and 3D printing system, in the field of medicine, particularly in renal interventions for cancer treatment.Methods:A specialized software transforms 2D medical images into precise 3D digital models, facilitating improved anatomical understanding and surgical planning. Patient-specific 3D printed anatomical models are utilized for preoperative planning, intraoperative guidance, and surgical education. AR technology enables the overlay of digital perceptions onto real-world surgical environments.Results:Patient-specific 3D printed anatomical models have multiple applications, such as preoperative planning, intraoperative guidance, trainee education, and patient counseling. Virtual reality involves substituting the real world with a computer-generated 3D environment, while AR overlays digitally created perceptions onto the existing reality. The advances in 3D modeling technology have sparked considerable interest in their application to partial nephrectomy in the realm of renal cancer. 3D printing, also known as additive manufacturing, constructs 3D objects based on computer-aided design or digital 3D models. Utilizing 3D-printed preoperative renal models provides benefits for surgical planning, offering a more reliable assessment of the tumor's relationship with vital anatomical structures and enabling better preparation for procedures. AR technology allows surgeons to visualize patient-specific renal anatomical structures and their spatial relationships with surrounding organs by projecting CT/MRI images onto a live laparoscopic video. Incorporating patient-specific 3D digital models into healthcare enhances best practice, resulting in improved patient care, increased patient satisfaction, and cost saving for the healthcare system.
基金Postdoctoral Fund of China (No. 2003034518), Fund of Health Bureau of Zhejiang Province (No. 2004B042), China
文摘This research studies the process of 3D reconstruction and dynamic concision based on 2D medical digital images using virtual reality modelling language (VRML) and JavaScript language, with a focus on how to realize the dynamic concision of 3D medical model with script node and sensor node in VRML. The 3D reconstruction and concision of body internal organs can be built with such high quality that they are better than those obtained from the traditional methods. With the function of dynamic concision, the VRML browser can offer better windows for man-computer interaction in real-time environment than ever before. 3D reconstruction and dynamic concision with VRML can be used to meet the requirement for the medical observation of 3D reconstruction and have a promising prospect in the fields of medical imaging.
基金the National Key Research and Development Program of China(2016YFB1001501)NSF of China(61672457)+1 种基金the Fundamental Research Funds for the Central Universities(2018FZA5011)Zhejiang University-SenseTime Joint Lab of 3D Vision.
文摘Although VSLAM/VISLAM has achieved great success,it is still difficult to quantitatively evaluate the localization results of different kinds of SLAM systems from the aspect of augmented reality due to the lack of an appropriate benchmark.For AR applications in practice,a variety of challenging situations(e.g.,fast motion,strong rotation,serious motion blur,dynamic interference)may be easily encountered since a home user may not carefully move the AR device,and the real environment may be quite complex.In addition,the frequency of camera lost should be minimized and the recovery from the failure status should be fast and accurate for good AR experience.Existing SLAM datasets/benchmarks generally only provide the evaluation of pose accuracy and their camera motions are somehow simple and do not fit well the common cases in the mobile AR applications.With the above motivation,we build a new visual-inertial dataset as well as a series of evaluation criteria for AR.We also review the existing monocular VSLAM/VISLAM approaches with detailed analyses and comparisons.Especially,we select 8 representative monocular VSLAM/VISLAM approaches/systems and quantitatively evaluate them on our benchmark.Our dataset,sample code and corresponding evaluation tools are available at the benchmark website http://www.zjucvg.net/eval-vislam/.
文摘Realism rendering methods of outdoor augmented reality(AR)is an interesting topic.Realism items in outdoor AR need advanced impacts like shadows,sunshine,and relations between unreal items.A few realistic rendering approaches were built to overcome this issue.Several of these approaches are not dealt with real-time rendering.However,the issue remains an active research topic,especially in outdoor rendering.This paper introduces a new approach to accomplish reality real-time outdoor rendering by considering the relation between items in AR regarding shadows in any place during daylight.The proposed method includes three principal stages that cover various outdoor AR rendering challenges.First,real shadow recognition was generated considering the sun’s location and the intensity of the shadow.The second step involves real shadow protection.Finally,we introduced a shadow production algorithm technique and shades through its impacts on unreal items in the AR.The selected approach’s target is providing a fast shadow recognition technique without affecting the system’s accuracy.It achieved an average accuracy of 95.1%and an area under the curve(AUC)of 92.5%.The outputs demonstrated that the proposed approach had enhanced the reality of outside AR rendering.The results of the proposed method outperformed other state-of-the-art rendering shadow techniques’outcomes.
文摘Implementing augmented reality(AR)and three-dimensional(3D)imaging within routine surgical practice is challenging.This requires considerable time,investment,and perseverance.Recently,the application of such technologies has been scrutinized within the context of urology across multiple settings,including preoperative planning,intraoperative diagnostics,training,and counseling.^(l)Specific attention has been paid to standard anterior robot-assisted radical prostatectomy(RARP),where AR has demonstrated promising results.
文摘Objective:To highlight the role of hyper accuracy three-dimensional(3D)reconstruction in facilitating surgical planning and guiding selective clamping during robot-assisted partial nephrectomy(RAPN).Methods:A transperitoneal RAPN was performed in a 62-year-old male patient presenting with a 4 cm right anterior interpolar renal mass(R.E.N.A.L nephrometry score 7A).An abnormal vasculature was observed,with a single renal vein and two right renal arteries originating superiorly to the vein and anterior,when dividing in their segmental branches.According to the hyper accuracy 3D(HA3D^(®))rainbow model(MEDICS Srl,Turin,Italy),one branch belonging to one of the segmental arteries was feeding the tumor.This allowed for an accurate prediction of the area vascularized by each arterial branch.The 3D model was included in the intraoperative console view during the whole procedure,using the TilePro feature.A step-by-step explanation of the procedure is provided in the video attached to the present article.Results:The operative time was 90 min with a warm ischemia time on selective clamping of 13 min.Estimated blood loss was 180 mL.No intraoperative complication was encountered and no drain was placed at the end of the procedure.The patient was discharged on postoperative Day 2,without any early postoperative complications.The final pathology report showed a pathological tumor stage 1 clear cell renal cell carcinoma with negative surgical margins.Conclusion:The present study and the attached video illustrate the value of 3D rainbow model during the planning and execution of a RAPN with selective clamping.It shows how the surgeon can rely on this model to be more efficient by avoiding unnecessary surgical steps,and to safely adopt a“selective”clamping strategy that can translate in minimal functional impact.
文摘The landscape of hepatobiliary surgical education has undergone a significanttransformation with the integration of advanced technologies such as threedimensionalmodeling,virtual reality,augmented reality,and artificial intelligence.This review synthesizes recent advancements in surgical education,examiningthe role of these technologies in improving anatomical understanding,surgicalskill acquisition,and overall trainee engagement.Evidence from randomizedcontrolled trials,systematic reviews,and cohort studies shows that immersivetraining tools,including virtual reality,augmented reality,and haptic feedback,outperform traditional apprenticeship methods in fostering cognitive and psychomotorskills.Artificial intelligence applications provide real-time feedback,furtherenhancing learning efficiency.However,these technologies should complement,rather than replace,traditional hands-on training.Some challenges remain to beaddressed,such as high costs,infrastructure requirements,and limited long-termvalidation of these technologies.The review concludes that while these innovationsoffer promising educational benefits,further research is needed to standardizetheir application and evaluate their long-term impact on surgical outcomes.
文摘Unlike earlier innovations that focused on manipulation and task precision,the latest surgical advancements emphasize data visualization and the acceleration of surgeons’decision-making.Before entering the operating room,digital tools help surgeons develop patient-specific approaches,outline procedural steps,and prepare strategies for managing potential crises.During the operation,relevant data appear at the right moment to assist decision-making and guide performance in formats tailored to the surgeon’s physical and cognitive state,all without causing distractions.In the future operating room,performance data from surgeons will be gathered,integrated,and analyzed to interpret surgeons’behavior and confidence.Advanced techniques,including artificial intelligence,can be employed to automatically detect moments of performance difficulty or cognitive overload.As a result,digital surgery provides timely,context-specific assistance to surgeons in high-stakes situations.Key components of digital surgery are highlighted in this narrative review.
