A virtual interventional surgical system with force feedback is designed to provide practice before complicated interventional operation and assistance during operation.The collision detection,vessel deformation calcu...A virtual interventional surgical system with force feedback is designed to provide practice before complicated interventional operation and assistance during operation.The collision detection,vessel deformation calculating and virtual force computing of the virtual system are implemented by using skeleton spring model as the physical modeling foundation,which is based on the mass spring model and easy to construct with high computing efficiency.In order to increase the real time performance,the central plane of the vessel model is extracted and then simplified to complete the skeleton filling.The initiative bending kinematics of the virtual catheter is analyzed so as to provide the virtual system with higher fidelity.The experimental results show that the virtual system can well simulate the vessel deformation and force feedback within an interventional surgery,which gives the virtual system better immersion.展开更多
Force feedback dataglove is an important interface of human-machine interaction between manipulator and virtual assembly system, which is in charge of the bidirectional transmission of movement and force information b...Force feedback dataglove is an important interface of human-machine interaction between manipulator and virtual assembly system, which is in charge of the bidirectional transmission of movement and force information between computer and operator. The exoskeleton force feedback dataglove is designed taking the pneumatic artificial muscle as actuator, meanwhile, its structure and work principle are introduced, and the force control problem is analyzed and researched by experiment. The mathematic model of grasping rigid object for finger is established. Considering the friction of tendon-sheath system and finger deformation, the closed-loop force control for a single joint, a single finger and multi-fingers are studied respectively by the feedforward proportional-integral（PI） control method with variable arguments. On the premise of the force smoothness, the control error of the force exerted on the finger joint is in the range of ±0.25 N, which meets the requirement of force feedback. By experimental analysis, the reason of force fluctuation is that the finger joint has a small amplitude quiver, and the consistent change tendency of the force between proximal interphalangeal（PIP） joint and distal interphalangeal（DIP） joint results from their angle coupling relationship.展开更多
Micromanipulation has been recognized to be very difficult due to the inefficiency oftraditional micromanipulation methods. The paper present a general framework formicromanipulation robot based on virtual reality tec...Micromanipulation has been recognized to be very difficult due to the inefficiency oftraditional micromanipulation methods. The paper present a general framework formicromanipulation robot based on virtual reality technology. The significance of in-troducing virtual reality into micromanipulation is analyzed, and the current researchin this field is reviewed. Based on this, we propose a micromanipulation system thatintegrates virtual environment with vision feedback and force feedback. The systemrealizes vision close-loop control and force close-loop control to enhance the perfor-mance of micromanipulation device. A graphics modelling method is proposed for amicroassembly task. Hardware and software implementation is described and discus-sion about the research is presented.展开更多
The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of inf...The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of information that is accepted and processed by the user＇s perception system. The increase of the useful feedback information may reduce the user＇s cognitive load, thus enhancing the user＇s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment （VAE） and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes： efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times （TCTs）, questionnaires, and human performance error rates （HPERs）, respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback （visual or auditory） or no feedback. The participants＇ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.展开更多
Most existing force feedback methods are still difficult to meet the requirements of real-time force calculation in virtual assembly and operation with complex objects. In addition, there is often an assumption that t...Most existing force feedback methods are still difficult to meet the requirements of real-time force calculation in virtual assembly and operation with complex objects. In addition, there is often an assumption that the controlled objects are completely flee and the target object is only completely fixed or flee, thus, the dynamics of the kinematic chain where the controlled objects are located are neglected during the physical simulation of the product manipulation with force feedback interaction. This paper proposes a physical simulation method of product assembly and operation manipulation based on statistically learned contact force prediction model and the coupling of force feedback and dynamics. In the proposed method, based on hidden Markov model （HMM） and local weighting learning （LWL）, contact force prediction model is constructed, which can estimate the contact force in real time during interaction. Based on computational load balance model, the computing resources are dynamically assigned and the dynamics integral step is optimized. In addition, smoothing process is performed to the force feedback on the synchronization points. Consequently, we can solve the coupling and synchronization problems of high-frequency feedback force servo. low-frequency dynamics solver servo and scene rendering servo, and realize highly stable and accurate force feedback in the physical simulation of product assembly and operation manipulation. This research proposes a physical simulation method of product assembly and operation manipulation.展开更多
Nano assembly and manipulation technologies are the basis for nano-electro-mechanical systems（NEMS）. Atomic force microscope（AFM） is widely used to manipulate nanotubes to assemble NEMS. Manipulating nanotubes wit...Nano assembly and manipulation technologies are the basis for nano-electro-mechanical systems（NEMS）. Atomic force microscope（AFM） is widely used to manipulate nanotubes to assemble NEMS. Manipulating nanotubes with AFM is a challenging and difficult task. One of the major reasons is the deficiency of visual information during the manipulation process. To address these difficulties, this research aims to put forward novel virtual tools and assembly strategies to improve the efficiency, accuracy and ease of the assembly process of NEMS. This paper begins by the discussion of the principles and implementation of a virtual nano-assembly simulator, which serves as a benchmark to test the proposed NEMS assembly techniques and virtual tools. Then, a general framework of nanotube-based NEMS assembly is proposed. Several nano-assembly strategies and virtual tools, such as automated path planning for NEMS assembly, a four-step scheme of nanotube manipulation, virtual fixtures for assembly finalization and safe manipulation, are introduced. These virtual tools and methods are experimented for justification. An assembly task of moderate complexity was performed in our virtual nano-assembly simulator with and without the help of the proposed toolkit. Experimental results suggest that the proposed methods tend to greatly enhance the efficiency and accuracy of nanotube-based NEMS assembly. In general, the proposed virtual reality toolkit not only ensures the safety, but also enhances the accuracy and efficiency of the assembly of nanotube-based NEMS.展开更多
基金supported by National High Technology Development Program of China(No. 51575256)
文摘A virtual interventional surgical system with force feedback is designed to provide practice before complicated interventional operation and assistance during operation.The collision detection,vessel deformation calculating and virtual force computing of the virtual system are implemented by using skeleton spring model as the physical modeling foundation,which is based on the mass spring model and easy to construct with high computing efficiency.In order to increase the real time performance,the central plane of the vessel model is extracted and then simplified to complete the skeleton filling.The initiative bending kinematics of the virtual catheter is analyzed so as to provide the virtual system with higher fidelity.The experimental results show that the virtual system can well simulate the vessel deformation and force feedback within an interventional surgery,which gives the virtual system better immersion.
基金supported by National Natural Science Foundation of China (Grant No. 50375034)Research Foundation for the Doctoral Program of Higher Education of China (Grant No. 200802881002)
文摘Force feedback dataglove is an important interface of human-machine interaction between manipulator and virtual assembly system, which is in charge of the bidirectional transmission of movement and force information between computer and operator. The exoskeleton force feedback dataglove is designed taking the pneumatic artificial muscle as actuator, meanwhile, its structure and work principle are introduced, and the force control problem is analyzed and researched by experiment. The mathematic model of grasping rigid object for finger is established. Considering the friction of tendon-sheath system and finger deformation, the closed-loop force control for a single joint, a single finger and multi-fingers are studied respectively by the feedforward proportional-integral（PI） control method with variable arguments. On the premise of the force smoothness, the control error of the force exerted on the finger joint is in the range of ±0.25 N, which meets the requirement of force feedback. By experimental analysis, the reason of force fluctuation is that the finger joint has a small amplitude quiver, and the consistent change tendency of the force between proximal interphalangeal（PIP） joint and distal interphalangeal（DIP） joint results from their angle coupling relationship.
基金supported by the National High Technology Research and Development Program of China (No.2002AA40445O)also supported by the National Defense Basic Scientific Research Foundation of China
文摘Micromanipulation has been recognized to be very difficult due to the inefficiency oftraditional micromanipulation methods. The paper present a general framework formicromanipulation robot based on virtual reality technology. The significance of in-troducing virtual reality into micromanipulation is analyzed, and the current researchin this field is reviewed. Based on this, we propose a micromanipulation system thatintegrates virtual environment with vision feedback and force feedback. The systemrealizes vision close-loop control and force close-loop control to enhance the perfor-mance of micromanipulation device. A graphics modelling method is proposed for amicroassembly task. Hardware and software implementation is described and discus-sion about the research is presented.
基金This work was supported in part by EPSRC(No.GR/M69333/01(P)).
文摘The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of information that is accepted and processed by the user＇s perception system. The increase of the useful feedback information may reduce the user＇s cognitive load, thus enhancing the user＇s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment （VAE） and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes： efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times （TCTs）, questionnaires, and human performance error rates （HPERs）, respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback （visual or auditory） or no feedback. The participants＇ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.
基金Supported by National Natural Science Foundation of China(51475418)National Basic Research 973 Program of China(2011CB706503)Science Fund for Creative Research Groups of National Natural Science Foundation of China(51221004)
文摘Most existing force feedback methods are still difficult to meet the requirements of real-time force calculation in virtual assembly and operation with complex objects. In addition, there is often an assumption that the controlled objects are completely flee and the target object is only completely fixed or flee, thus, the dynamics of the kinematic chain where the controlled objects are located are neglected during the physical simulation of the product manipulation with force feedback interaction. This paper proposes a physical simulation method of product assembly and operation manipulation based on statistically learned contact force prediction model and the coupling of force feedback and dynamics. In the proposed method, based on hidden Markov model （HMM） and local weighting learning （LWL）, contact force prediction model is constructed, which can estimate the contact force in real time during interaction. Based on computational load balance model, the computing resources are dynamically assigned and the dynamics integral step is optimized. In addition, smoothing process is performed to the force feedback on the synchronization points. Consequently, we can solve the coupling and synchronization problems of high-frequency feedback force servo. low-frequency dynamics solver servo and scene rendering servo, and realize highly stable and accurate force feedback in the physical simulation of product assembly and operation manipulation. This research proposes a physical simulation method of product assembly and operation manipulation.
基金supported by National Important Project on Science&Technology of China(Grant No.2009ZX04014-031)Specialized Research Fund for the Doctoral Program of Higher Education of China(Grant No.200803350031)+2 种基金Zhejiang Provincial Natural Science Foundation of China(Grant No.Y1080358)Research Fund of Zhejiang Educational Committee of China(Grant No.Y200909651)PACMAN Project within the French National Research Agency
文摘Nano assembly and manipulation technologies are the basis for nano-electro-mechanical systems（NEMS）. Atomic force microscope（AFM） is widely used to manipulate nanotubes to assemble NEMS. Manipulating nanotubes with AFM is a challenging and difficult task. One of the major reasons is the deficiency of visual information during the manipulation process. To address these difficulties, this research aims to put forward novel virtual tools and assembly strategies to improve the efficiency, accuracy and ease of the assembly process of NEMS. This paper begins by the discussion of the principles and implementation of a virtual nano-assembly simulator, which serves as a benchmark to test the proposed NEMS assembly techniques and virtual tools. Then, a general framework of nanotube-based NEMS assembly is proposed. Several nano-assembly strategies and virtual tools, such as automated path planning for NEMS assembly, a four-step scheme of nanotube manipulation, virtual fixtures for assembly finalization and safe manipulation, are introduced. These virtual tools and methods are experimented for justification. An assembly task of moderate complexity was performed in our virtual nano-assembly simulator with and without the help of the proposed toolkit. Experimental results suggest that the proposed methods tend to greatly enhance the efficiency and accuracy of nanotube-based NEMS assembly. In general, the proposed virtual reality toolkit not only ensures the safety, but also enhances the accuracy and efficiency of the assembly of nanotube-based NEMS.
