The shop floor dynamic scheduling system based on human-computer interaction is the use of computer-aided decision-making and human-computer interaction to solve the dynamic scheduling problem.A human-computer interac...The shop floor dynamic scheduling system based on human-computer interaction is the use of computer-aided decision-making and human-computer interaction to solve the dynamic scheduling problem.A human-computer interaction interface based on Gantt chart is designed,which can not only comprehensively and quantitatively represent the scheduling process and scheduling scheme,but also have friendly human-computer interaction performance.The data transmission and interaction architecture is constructed to realize the rapid response to shop floor disturbance events.A priority calculation algorithm integrating priority rules and dispatcher preference is proposed,which realizes the automatic calculation of priority for the dispatcher's reference and reduces theirburden.A man-machine interactive shop floor dynamic scheduling strategy is proposed.When solving the dynamic flexible job shop scheduling problem caused by machine tool breakdown and urgent order,the origin moments obtained by using this strategy are 0.4190 and 0.3703 respectively.As can be seen from the origin moment indicator,the dynamic shop floor scheduling system based on the human-computer interaction is efficient and reliable in solving dynamic scheduling problems,and related strategies of this system are also feasible and stable.展开更多
Rock discontinuities control rock mechanical behaviors and significantly influence the stability of rock masses.However,existing discontinuity mapping algorithms are susceptible to noise,and the calculation results ca...Rock discontinuities control rock mechanical behaviors and significantly influence the stability of rock masses.However,existing discontinuity mapping algorithms are susceptible to noise,and the calculation results cannot be fed back to users timely.To address this issue,we proposed a human-machine interaction(HMI)method for discontinuity mapping.Users can help the algorithm identify the noise and make real-time result judgments and parameter adjustments.For this,a regular cube was selected to illustrate the workflows:(1)point cloud was acquired using remote sensing;(2)the HMI method was employed to select reference points and angle thresholds to detect group discontinuity;(3)individual discontinuities were extracted from the group discontinuity using a density-based cluster algorithm;and(4)the orientation of each discontinuity was measured based on a plane fitting algorithm.The method was applied to a well-studied highway road cut and a complex natural slope.The consistency of the computational results with field measurements demonstrates its good accuracy,and the average error in the dip direction and dip angle for both cases was less than 3.Finally,the computational time of the proposed method was compared with two other popular algorithms,and the reduction in computational time by tens of times proves its high computational efficiency.This method provides geologists and geological engineers with a new idea to map rapidly and accurately rock structures under large amounts of noises or unclear features.展开更多
The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, ...The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, elements and contents for the Web GIS. The design of the Wuhan Bus Search System is fulfilled to confirm the validity and practicability of the fusion.展开更多
Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Severa...Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.展开更多
Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgen...Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.展开更多
Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriousl...Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.展开更多
Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation...Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.展开更多
Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,...Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.展开更多
As the Internet of Things advances,gesture recognition emerges as a prominent domain in human-machine interaction(HMI).However,interactive wearables based on conductive hydrogels for individuals with single-arm functi...As the Internet of Things advances,gesture recognition emerges as a prominent domain in human-machine interaction(HMI).However,interactive wearables based on conductive hydrogels for individuals with single-arm functionality or disabilities remain underexplored.Here,we devised a wearable one-handed keyboard with gesture recognition,employing machine learning algorithms and hydrogel-based mechanical sensors to boost productivity.PCG(PAM/CMC/rGO)hydrogels are composed of polyacrylamide(PAM),sodium carboxymethyl cellulose(CMC),and reduced graphene oxide(rGO),which function as a strain,pressure sensor,and electrode material.The PAM chains offer the gel’s elasticity by covalent cross-linking,while the biocompatible CMC improves the dispersion of rGO and promotes electromechanical properties.Integrating rGO sheets into the polymer matrix facilitates cross-linking and generates supple-mentary conductive pathways,thereby augmenting the gel system’s elasticity,sensitivity,and durability.Our hydrogel sensors include high sensitivity(gage factor(GF)=8.18,395.6%-551.96%)and superior pressure sensing capabilities(Sensitivity(S)=0.3116 kPa^(-1),0-9.82 kPa).Furthermore,we developed a wearable keyboard with up to 98.13%accuracy using convolutional neural networks and a custom data acquisition system.This study establishes the groundwork for creating multifunctional gel sensors for intelligent machines,wearable devices,and brain-computer interfaces.展开更多
Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However...Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However, the current systems should take advantage of the operator's attention to obtain the optimal solution.In this paper, we innovatively propose a human-machine collaborative support scheduling system of intelligence information from multi-UAVs based on eye-tracker. Firstly, the target recognition algorithm is applied to the images from the multiple unmanned aerial vehicles(multi-UAVs) to recognize the targets in the images. Then,the support system utilizes the eye tracker to gain the eye-gaze points which are intended to obtain the focused targets in the images. Finally, the heuristic scheduling algorithms take both the attributes of targets and the operator's attention into consideration to obtain the sequence of the images. As the processing time of the images collected by the multi-UAVs is uncertain, however the upper bounds and lower bounds of the processing time are known before. So the processing time of the images is modeled by the interval processing time. The objective of the scheduling problem is to minimize mean weighted completion time. This paper proposes some new polynomial time heuristic scheduling algorithms which firstly schedule the images including the focused targets. We conduct the scheduling experiments under six different distributions. The results indicate that the proposed algorithm is not sensitive to the different distributions of the processing time and has a negligible computational time. The absolute error of the best performing heuristic solution is only about 1%. Then, we incorporate the best performing heuristic algorithm into the human-machine collaborative support systems to verify the performance of the system.展开更多
In view of the current gesture recognition algorithm based on skin color segmentation is not flexible and has weak resistance to the environment, this paper puts forward a new method of skin color modeling to improve ...In view of the current gesture recognition algorithm based on skin color segmentation is not flexible and has weak resistance to the environment, this paper puts forward a new method of skin color modeling to improve the adaptability of gesture segmentation when it face to different states. The modeling built by double color space instead of only one is compatible both in YCbCr and HSV color space to training the Gaussian model which can update the threshold value for binarization. Finally, this paper designed a natural gesture recognition and interactive systems based on the double color space model. It has shown that the system has a good interactive experience in different environments.展开更多
Triboelectric nanogenerators(TENGs)represent a promising technology for next generation human–computer interaction.The effective enhancement of induced charges are critical factors that determine the recognition accu...Triboelectric nanogenerators(TENGs)represent a promising technology for next generation human–computer interaction.The effective enhancement of induced charges are critical factors that determine the recognition accuracy of TENGbased tactile sensors.Here,we propose a magnetic fieldassisted TENG device utilizing waveform feature enrichment strategies to significantly enhance the tactile recognition accuracy in natural environments.An elastic micro-nano structure was fabricated on a polydimethylsiloxane(PDMS)film via a facile templating method.Leveraging the inherent hydrophobicity and microscale surface roughness of PDMS,our device demonstrates stable and distinct waveform characteristics under natural operating conditions.Importantly,the introduction of a magnetic field generates a Lorentz force,which effectively modulates induced charges within the electrode,yet minimally affects triboelectric charges at the PDMS interface.This selective modulation induces an asymmetric charge distribution inside the electrode,substantially increasing the induced charge density,consequently,subtle waveform features are markedly enhanced.These enriched signal features play a crucial role in elevating material recognition accuracy.As a result,the sensor achieves a remarkable recognition accuracy of 99%when distinguishing among ten different materials under magnetic field assistance.This work provides valuable guidelines for advancing the performance and accuracy of TENG-based tactile sensing systems.展开更多
1 Introduction Amid escalating global climate change,the“dual carbon”goals of carbon peak and carbon neutrality have become a focal point of global attention and an important strategy for sustainable development[1]....1 Introduction Amid escalating global climate change,the“dual carbon”goals of carbon peak and carbon neutrality have become a focal point of global attention and an important strategy for sustainable development[1].With the rapid development of renewable energy technologies and the increasing public demand for environmental protection and low-carbon living,the adoption of new energy vehicles,particularly electric vehicles(EVs).展开更多
Porous hydrogel sensors have attracted significant attention in fields such as smart wearables and medical monitoring due to their high sensitivity.However,existing fabrication methods typically degrade the surface sm...Porous hydrogel sensors have attracted significant attention in fields such as smart wearables and medical monitoring due to their high sensitivity.However,existing fabrication methods typically degrade the surface smoothness of hydrogels when introducing porous structures and face significant challenges in removing fillers completely.To address these challenges,we herein introduce a novel one-step,thermosensitive spray-coating technique for the preparation of aircell hydrogel(ACH).This method leverages the rapid cooling of a thermoresponsive gelatin methacryloyl solution through atomization,enabling rapid cross-linking within seconds and air bubbles encapsulated in situ.Additionally,the transient flow of the pre-gel facilitates the repair of voids formed by ruptured surface bubbles,leading to the creation of the ACH with uniformly distributed inner air bubbles and a smooth outer surface.The mold-free fabrication method is independent of substrate surface properties,enabling the creation of a porous hydrogel film with a thickness as thin as 163??m.Furthermore,the dual-crosslinked network endows the ACH with excellent anti-swelling properties,and the physical crosslinking between gelatin molecules allows the ACH to self-heal.The ACH exhibits excellent sensitivity in deformation sensing and can even successfully track minor external forces,which enables it to effectively complete various tasks such as facial expression recognition,pitch differentiation,and motion detection.By integrating the ACH into a sensing glove,we also demonstrate the significant potential of the ACH for applications in human-machine interaction and tactile sensing.Ultimately,the ACH sensors are also applied to motion mapping and machine tactile feedback,indicating their promising potential in human-machine interaction.展开更多
This study presents a hybrid CNN-Transformer model for real-time recognition of affective tactile biosignals.The proposed framework combines convolutional neural networks(CNNs)to extract spatial and local temporal fea...This study presents a hybrid CNN-Transformer model for real-time recognition of affective tactile biosignals.The proposed framework combines convolutional neural networks(CNNs)to extract spatial and local temporal features with the Transformer encoder that captures long-range dependencies in time-series data through multi-head attention.Model performance was evaluated on two widely used tactile biosignal datasets,HAART and CoST,which contain diverse affective touch gestures recorded from pressure sensor arrays.TheCNN-Transformer model achieved recognition rates of 93.33%on HAART and 80.89%on CoST,outperforming existing methods on both benchmarks.By incorporating temporal windowing,the model enables instantaneous prediction,improving generalization across gestures of varying duration.These results highlight the effectiveness of deep learning for tactile biosignal processing and demonstrate the potential of theCNN-Transformer approach for future applications in wearable sensors,affective computing,and biomedical monitoring.展开更多
Surgical robots are designed to provide enhanced precision and dexterity compared to manual surgical procedures,which mainly rely on multimodal sensing technologies for the surgeon to seamlessly operate the robotic ar...Surgical robots are designed to provide enhanced precision and dexterity compared to manual surgical procedures,which mainly rely on multimodal sensing technologies for the surgeon to seamlessly operate the robotic arms and instruments.Compared with single-mode sensors,optical and mechanical bi-modal sensors provide improved precision,enhanced safety,and robustness of human-machine interaction systems.Here,the template-guided and pneumatic printing technologies are combined to construct perovskite and graphene parallel structures with both optical and mechanical sensing capabilities.The printed uniformly crystallized perovskite microstructure exhibits fast and sensitive photoelectric response characteristics,enabling shadow recognition functionality.The combination of graphene and elastic rubber endows the great printability to prepare parallel structures near the perovskite arrays for force sensing capabilities.Thus,the printed perovskite and graphene structures possess non-contact optical sensing capabilities to detect hand position by recognizing shadows between the hand and the sensor,as well as contact mechanical sensing capabilities to detect touch force applied by the hand.It provides a synergistic platform for real-time and multidimensional feedback to improve human-machine interaction.展开更多
A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In thi...A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.展开更多
In human-machine interaction,robotic hands are useful in many scenarios.To operate robotic hands via gestures instead of handles will greatly improve the convenience and intuition of human-machine interaction.Here,we ...In human-machine interaction,robotic hands are useful in many scenarios.To operate robotic hands via gestures instead of handles will greatly improve the convenience and intuition of human-machine interaction.Here,we present a magnetic array assisted sliding triboelectric sensor for achieving a real-time gesture interaction between a human hand and robotic hand.With a finger’s traction movement of flexion or extension,the sensor can induce positive/negative pulse signals.Through counting the pulses in unit time,the degree,speed,and direction of finger motion can be judged in realtime.The magnetic array plays an important role in generating the quantifiable pulses.The designed two parts of magnetic array can transform sliding motion into contact-separation and constrain the sliding pathway,respectively,thus improve the durability,low speed signal amplitude,and stability of the system.This direct quantization approach and optimization of wearable gesture sensor provide a new strategy for achieving a natural,intuitive,and real-time human-robotic interaction.展开更多
Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tu...Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tunable mechanical properties by a physical cross-linking salting-freezing-thawing method.The gelling agent(β-Glycerophosphate sodium:Gp)induces the aggregation and binding of PVA molecular chains and thereby toughens them(stress up to 5.79 MPa,toughness up to 13.96 MJ m^(−3)).Notably,due to molecular self-assembly,hydrogels can be fully recycled and reprocessed by direct heating(100°C for a few seconds),and the tensile strength can still be maintained at about 100%after six recoveries.The hydrogel integrates transparency(>60%),super toughness(up to 13.96 MJ m^(−3),bearing 1500 times of its own tensile weight),good antibacterial properties(E.coli and S.aureus),UV protection(Filtration:80%–90%),high electrical conductivity(4.72 S m^(−1)),anti-swelling and recyclability.The hydrogel can not only monitor daily physiological activities,but also be used for complex activities underwater and message encryption/decryption.We also used it to create a complete finger joint rehabilitation system with an interactive interface that dynamically presents the user’s health status.Our multifunctional electronic skin will have a profound impact on the future of new rehabilitation medical,human–machine interaction,VR/AR and the metaverse fields.展开更多
基金supported by the Tianjin Enterprise Science and Technology Commissioner Project(Grant No.23YDTPJC00740,Grant No.24YDTPJC00610)the Tianjin Tiankai Higher Education Science and Technology Innovation Park Enterprise R&D Special Project(Grant No.23YFZXYC00027).
文摘The shop floor dynamic scheduling system based on human-computer interaction is the use of computer-aided decision-making and human-computer interaction to solve the dynamic scheduling problem.A human-computer interaction interface based on Gantt chart is designed,which can not only comprehensively and quantitatively represent the scheduling process and scheduling scheme,but also have friendly human-computer interaction performance.The data transmission and interaction architecture is constructed to realize the rapid response to shop floor disturbance events.A priority calculation algorithm integrating priority rules and dispatcher preference is proposed,which realizes the automatic calculation of priority for the dispatcher's reference and reduces theirburden.A man-machine interactive shop floor dynamic scheduling strategy is proposed.When solving the dynamic flexible job shop scheduling problem caused by machine tool breakdown and urgent order,the origin moments obtained by using this strategy are 0.4190 and 0.3703 respectively.As can be seen from the origin moment indicator,the dynamic shop floor scheduling system based on the human-computer interaction is efficient and reliable in solving dynamic scheduling problems,and related strategies of this system are also feasible and stable.
基金supported by the National Key R&D Program of China(No.2023YFC3081200)the National Natural Science Foundation of China(No.42077264)the Scientific Research Project of PowerChina Huadong Engineering Corporation Limited(HDEC-2022-0301).
文摘Rock discontinuities control rock mechanical behaviors and significantly influence the stability of rock masses.However,existing discontinuity mapping algorithms are susceptible to noise,and the calculation results cannot be fed back to users timely.To address this issue,we proposed a human-machine interaction(HMI)method for discontinuity mapping.Users can help the algorithm identify the noise and make real-time result judgments and parameter adjustments.For this,a regular cube was selected to illustrate the workflows:(1)point cloud was acquired using remote sensing;(2)the HMI method was employed to select reference points and angle thresholds to detect group discontinuity;(3)individual discontinuities were extracted from the group discontinuity using a density-based cluster algorithm;and(4)the orientation of each discontinuity was measured based on a plane fitting algorithm.The method was applied to a well-studied highway road cut and a complex natural slope.The consistency of the computational results with field measurements demonstrates its good accuracy,and the average error in the dip direction and dip angle for both cases was less than 3.Finally,the computational time of the proposed method was compared with two other popular algorithms,and the reduction in computational time by tens of times proves its high computational efficiency.This method provides geologists and geological engineers with a new idea to map rapidly and accurately rock structures under large amounts of noises or unclear features.
基金Supported by the National Natural Science Foundation of China （No. 40071071）.
文摘The fusion of VlSI （visual identity system Internet）, digital maps and Web GIS is presented. Web GIS interface interactive design with VISI needs to consider more new factors. VISI can provide the design principle, elements and contents for the Web GIS. The design of the Wuhan Bus Search System is fulfilled to confirm the validity and practicability of the fusion.
基金Supported by the‘Automotive Glazing Application in Intelligent Cockpit Human-Machine Interface’project(SKHX2021049)a collaboration between the Saint-Go Bain Research and the Beijing Normal University。
文摘Background With an increasing number of vehicles becoming autonomous,intelligent,and connected,paying attention to the future usage of car human-machine interface with these vehicles should become more relevant.Several studies have addressed car HMI but were less attentive to designing and implementing interactive glazing for every day(autonomous)driving contexts.Methods Reflecting on the literature,we describe an engineering psychology practice and the design of six novel future user scenarios,which envision the application of a specific set of augmented reality(AR)support user interactions.Additionally,we conduct evaluations on specific scenarios and experiential prototypes,which reveal that these AR scenarios aid the target user groups in experiencing a new type of interaction.The overall evaluation is positive with valuable assessment results and suggestions.Conclusions This study can interest applied psychology educators who aspire to teach how AR can be operationalized in a human-centered design process to students with minimal pre-existing expertise or minimal scientific knowledge in engineering psychology.
基金supported in part by the National Natural Science Foundation of China(U181321461773369+2 种基金61903360)the Selfplanned Project of the State Key Laboratory of Robotics(2020-Z12)China Postdoctoral Science Foundation funded project(2019M661155)。
文摘Electromyography(EMG)has already been broadly used in human-machine interaction(HMI)applications.Determining how to decode the information inside EMG signals robustly and accurately is a key problem for which we urgently need a solution.Recently,many EMG pattern recognition tasks have been addressed using deep learning methods.In this paper,we analyze recent papers and present a literature review describing the role that deep learning plays in EMG-based HMI.An overview of typical network structures and processing schemes will be provided.Recent progress in typical tasks such as movement classification,joint angle prediction,and force/torque estimation will be introduced.New issues,including multimodal sensing,inter-subject/inter-session,and robustness toward disturbances will be discussed.We attempt to provide a comprehensive analysis of current research by discussing the advantages,challenges,and opportunities brought by deep learning.We hope that deep learning can aid in eliminating factors that hinder the development of EMG-based HMI systems.Furthermore,possible future directions will be presented to pave the way for future research.
基金financially supported by the Natural Science Foundation of China(Nos.22109120,62104170 and 82202757)Zhejiang Provincial Natural Science Foundation of China(Nos.LQ21B030002 and LY23F040001)。
文摘Hydrogel-based triboelectric nanoge nerator(TENG)has a promising applied prospect in wearable electronic devices.However,its low performance,poor stability,insufficient recyclability and inferior self-healing seriously hinder its development.Herein,we report a robust route to a liquid metal(LM)/polyvinyl alcohol(PVA)hydrogel-based TENG(LP-TENG).Owing to the intrinsically liquid feature of conductive LM within the flexible PVA hydrogel,the as-prepared LP-TENG exhibited comprehensiye advantages of adaptability,biocompatibility,outstanding electrical performance,superior stability,recyclability and diverse applications,which were unattainable by traditional systems.Concretely,the LP-TENG delivered appealing open circuit voltage of 250 V,short circuit current of 4μA and transferred charge of 120 nC with high stability,outperforming most advanced TENG systems.The LP-TENG was successfully employed for versatile applications with multifunctionality,including human motion detection,handwriting recognition,energy collection,message transmission and human-machine interaction.This work presents significant prospects for crafting advanced materials and devices in the fields of wearable electronics,flexible skin and smart robots.
文摘Speech recognition rate will deteriorate greatly in human-machine interaction when the speaker's speech mixes with a bystander's voice. This paper proposes a time-frequency approach for Blind Source Seperation (BSS) for intelligent Human-Machine Interaction(HMI). Main idea of the algorithm is to simultaneously diagonalize the correlation matrix of the pre-whitened signals at different time delays for every frequency bins in time-frequency domain. The prososed method has two merits: (1) fast convergence speed; (2) high signal to interference ratio of the separated signals. Numerical evaluations are used to compare the performance of the proposed algorithm with two other deconvolution algorithms. An efficient algorithm to resolve permutation ambiguity is also proposed in this paper. The algorithm proposed saves more than 10% of computational time with properly selected parameters and achieves good performances for both simulated convolutive mixtures and real room recorded speeches.
基金NSFC-Shenzhen Robotics Research Center Project(No.U2013207)the Beijing Science and Technology Plan Project(No.Z191100008019008)。
文摘Teleoperation is of great importance in the area of robotics,especially when people are unavailable in the robot workshop.It provides a way for people to control robots remotely using human intelligence.In this paper,a robotic teleoperation system for precise robotic manipulation is established.The data glove and the 7-degrees of freedom(DOFs)force feedback controller are used for the remote control interaction.The control system and the monitor system are designed for the remote precise manipulation.The monitor system contains an image acquisition system and a human-machine interaction module,and aims to simulate and detect the robot running state.Besides,a visual object tracking algorithm is developed to estimate the states of the dynamic system from noisy observations.The established robotic teleoperation systemis applied to a series of experiments,and high-precision results are obtained,showing the effectiveness of the physical system.
基金supported by the China Postdoctoral Science Foundation(No.2022BG011)the Fundamental Research Funds for Central Universities(No.2020CDJ-LHZZ-077)+1 种基金the Natural Science Foundation of Chongqing,China(No.c stc2020jcyj-msxmX0397)the Fundamental Research Funds for Central Universities(No.00007717).
文摘As the Internet of Things advances,gesture recognition emerges as a prominent domain in human-machine interaction(HMI).However,interactive wearables based on conductive hydrogels for individuals with single-arm functionality or disabilities remain underexplored.Here,we devised a wearable one-handed keyboard with gesture recognition,employing machine learning algorithms and hydrogel-based mechanical sensors to boost productivity.PCG(PAM/CMC/rGO)hydrogels are composed of polyacrylamide(PAM),sodium carboxymethyl cellulose(CMC),and reduced graphene oxide(rGO),which function as a strain,pressure sensor,and electrode material.The PAM chains offer the gel’s elasticity by covalent cross-linking,while the biocompatible CMC improves the dispersion of rGO and promotes electromechanical properties.Integrating rGO sheets into the polymer matrix facilitates cross-linking and generates supple-mentary conductive pathways,thereby augmenting the gel system’s elasticity,sensitivity,and durability.Our hydrogel sensors include high sensitivity(gage factor(GF)=8.18,395.6%-551.96%)and superior pressure sensing capabilities(Sensitivity(S)=0.3116 kPa^(-1),0-9.82 kPa).Furthermore,we developed a wearable keyboard with up to 98.13%accuracy using convolutional neural networks and a custom data acquisition system.This study establishes the groundwork for creating multifunctional gel sensors for intelligent machines,wearable devices,and brain-computer interfaces.
基金the National Natural Science Foundation of China(No.61403410)
文摘Many human-machine collaborative support scheduling systems are used to aid human decision making by providing several optimal scheduling algorithms that do not take operator's attention into consideration.However, the current systems should take advantage of the operator's attention to obtain the optimal solution.In this paper, we innovatively propose a human-machine collaborative support scheduling system of intelligence information from multi-UAVs based on eye-tracker. Firstly, the target recognition algorithm is applied to the images from the multiple unmanned aerial vehicles(multi-UAVs) to recognize the targets in the images. Then,the support system utilizes the eye tracker to gain the eye-gaze points which are intended to obtain the focused targets in the images. Finally, the heuristic scheduling algorithms take both the attributes of targets and the operator's attention into consideration to obtain the sequence of the images. As the processing time of the images collected by the multi-UAVs is uncertain, however the upper bounds and lower bounds of the processing time are known before. So the processing time of the images is modeled by the interval processing time. The objective of the scheduling problem is to minimize mean weighted completion time. This paper proposes some new polynomial time heuristic scheduling algorithms which firstly schedule the images including the focused targets. We conduct the scheduling experiments under six different distributions. The results indicate that the proposed algorithm is not sensitive to the different distributions of the processing time and has a negligible computational time. The absolute error of the best performing heuristic solution is only about 1%. Then, we incorporate the best performing heuristic algorithm into the human-machine collaborative support systems to verify the performance of the system.
文摘In view of the current gesture recognition algorithm based on skin color segmentation is not flexible and has weak resistance to the environment, this paper puts forward a new method of skin color modeling to improve the adaptability of gesture segmentation when it face to different states. The modeling built by double color space instead of only one is compatible both in YCbCr and HSV color space to training the Gaussian model which can update the threshold value for binarization. Finally, this paper designed a natural gesture recognition and interactive systems based on the double color space model. It has shown that the system has a good interactive experience in different environments.
基金supports by the Guangxi Science and Technology Base and Talent Special Project(No.AD25069020)Major project of scientific research in colleges and universities of Anhui Province(No.2024AH040211)+3 种基金Department of Education team project(No.2022AH010096)Anhui Province key research and development program(Nos.202423i08050016 and 202304a05020027)the National Natural Science Foundation of China(No.52472190)Beijing Nova Program(Nos.20220484036 and 20240484565).
文摘Triboelectric nanogenerators(TENGs)represent a promising technology for next generation human–computer interaction.The effective enhancement of induced charges are critical factors that determine the recognition accuracy of TENGbased tactile sensors.Here,we propose a magnetic fieldassisted TENG device utilizing waveform feature enrichment strategies to significantly enhance the tactile recognition accuracy in natural environments.An elastic micro-nano structure was fabricated on a polydimethylsiloxane(PDMS)film via a facile templating method.Leveraging the inherent hydrophobicity and microscale surface roughness of PDMS,our device demonstrates stable and distinct waveform characteristics under natural operating conditions.Importantly,the introduction of a magnetic field generates a Lorentz force,which effectively modulates induced charges within the electrode,yet minimally affects triboelectric charges at the PDMS interface.This selective modulation induces an asymmetric charge distribution inside the electrode,substantially increasing the induced charge density,consequently,subtle waveform features are markedly enhanced.These enriched signal features play a crucial role in elevating material recognition accuracy.As a result,the sensor achieves a remarkable recognition accuracy of 99%when distinguishing among ten different materials under magnetic field assistance.This work provides valuable guidelines for advancing the performance and accuracy of TENG-based tactile sensing systems.
基金supported by Yunnan Provincial Basic Research Project(202401AT070344)National Natural Science Foundation of China(62263014).
文摘1 Introduction Amid escalating global climate change,the“dual carbon”goals of carbon peak and carbon neutrality have become a focal point of global attention and an important strategy for sustainable development[1].With the rapid development of renewable energy technologies and the increasing public demand for environmental protection and low-carbon living,the adoption of new energy vehicles,particularly electric vehicles(EVs).
基金financially supported by the National Key R&D Program of China(Grant No.2023YFE0108900)EU HORIZON 2021 L4DNANO(No.101086227)。
文摘Porous hydrogel sensors have attracted significant attention in fields such as smart wearables and medical monitoring due to their high sensitivity.However,existing fabrication methods typically degrade the surface smoothness of hydrogels when introducing porous structures and face significant challenges in removing fillers completely.To address these challenges,we herein introduce a novel one-step,thermosensitive spray-coating technique for the preparation of aircell hydrogel(ACH).This method leverages the rapid cooling of a thermoresponsive gelatin methacryloyl solution through atomization,enabling rapid cross-linking within seconds and air bubbles encapsulated in situ.Additionally,the transient flow of the pre-gel facilitates the repair of voids formed by ruptured surface bubbles,leading to the creation of the ACH with uniformly distributed inner air bubbles and a smooth outer surface.The mold-free fabrication method is independent of substrate surface properties,enabling the creation of a porous hydrogel film with a thickness as thin as 163??m.Furthermore,the dual-crosslinked network endows the ACH with excellent anti-swelling properties,and the physical crosslinking between gelatin molecules allows the ACH to self-heal.The ACH exhibits excellent sensitivity in deformation sensing and can even successfully track minor external forces,which enables it to effectively complete various tasks such as facial expression recognition,pitch differentiation,and motion detection.By integrating the ACH into a sensing glove,we also demonstrate the significant potential of the ACH for applications in human-machine interaction and tactile sensing.Ultimately,the ACH sensors are also applied to motion mapping and machine tactile feedback,indicating their promising potential in human-machine interaction.
文摘This study presents a hybrid CNN-Transformer model for real-time recognition of affective tactile biosignals.The proposed framework combines convolutional neural networks(CNNs)to extract spatial and local temporal features with the Transformer encoder that captures long-range dependencies in time-series data through multi-head attention.Model performance was evaluated on two widely used tactile biosignal datasets,HAART and CoST,which contain diverse affective touch gestures recorded from pressure sensor arrays.TheCNN-Transformer model achieved recognition rates of 93.33%on HAART and 80.89%on CoST,outperforming existing methods on both benchmarks.By incorporating temporal windowing,the model enables instantaneous prediction,improving generalization across gestures of varying duration.These results highlight the effectiveness of deep learning for tactile biosignal processing and demonstrate the potential of theCNN-Transformer approach for future applications in wearable sensors,affective computing,and biomedical monitoring.
基金supported by the National Natural Science Foundation of China(Grant Nos.52222313,22075296,52321006,T2394480,and T2394484)the National Key R&D Program of China(Grant Nos.2023YFE0111500,2021YFB3200701,and 2022YFB4700804)+1 种基金Beijing National Laboratory for Molecular Sciences(Grant No.BNLMSCXXM-202005)Beijing Municipal Science&Technology Commission(Grant No.Z231100005923039).
文摘Surgical robots are designed to provide enhanced precision and dexterity compared to manual surgical procedures,which mainly rely on multimodal sensing technologies for the surgeon to seamlessly operate the robotic arms and instruments.Compared with single-mode sensors,optical and mechanical bi-modal sensors provide improved precision,enhanced safety,and robustness of human-machine interaction systems.Here,the template-guided and pneumatic printing technologies are combined to construct perovskite and graphene parallel structures with both optical and mechanical sensing capabilities.The printed uniformly crystallized perovskite microstructure exhibits fast and sensitive photoelectric response characteristics,enabling shadow recognition functionality.The combination of graphene and elastic rubber endows the great printability to prepare parallel structures near the perovskite arrays for force sensing capabilities.Thus,the printed perovskite and graphene structures possess non-contact optical sensing capabilities to detect hand position by recognizing shadows between the hand and the sensor,as well as contact mechanical sensing capabilities to detect touch force applied by the hand.It provides a synergistic platform for real-time and multidimensional feedback to improve human-machine interaction.
文摘A complete characterization of the behavior in human-robot interactions(HRI) includes both: the behavioral dynamics and the control laws that characterize how the behavior is regulated with the perception data. In this way, this work proposes a leader-follower coordinate control based on an impedance control that allows to establish a dynamic relation between social forces and motion error. For this, a scheme is presented to identify the impedance based on fictitious social forces, which are described by distance-based potential fields.As part of the validation procedure, we present an experimental comparison to select the better of two different fictitious force structures. The criteria are determined by two qualities: least impedance errors during the validation procedure and least parameter variance during the recursive estimation procedure.Finally, with the best fictitious force and its identified impedance,an impedance control is designed for a mobile robot Pioneer 3AT,which is programmed to follow a human in a structured scenario.According to results, and under the hypothesis that moving like humans will be acceptable by humans, it is believed that the proposed control improves the social acceptance of the robot for this kind of interaction.
基金This work was supported by National Natural Science Foundation of China(51902035 and 52073037)Natural Science Foundation of Chongqing(cstc2020jcyj-msxmX0807)+1 种基金the Fundamental Research Funds for the Central Universities(2020CDJ-LHSS-001 and 2019CDXZWL001)Chongqing graduate tutor team construction project(ydstd1832).
文摘In human-machine interaction,robotic hands are useful in many scenarios.To operate robotic hands via gestures instead of handles will greatly improve the convenience and intuition of human-machine interaction.Here,we present a magnetic array assisted sliding triboelectric sensor for achieving a real-time gesture interaction between a human hand and robotic hand.With a finger’s traction movement of flexion or extension,the sensor can induce positive/negative pulse signals.Through counting the pulses in unit time,the degree,speed,and direction of finger motion can be judged in realtime.The magnetic array plays an important role in generating the quantifiable pulses.The designed two parts of magnetic array can transform sliding motion into contact-separation and constrain the sliding pathway,respectively,thus improve the durability,low speed signal amplitude,and stability of the system.This direct quantization approach and optimization of wearable gesture sensor provide a new strategy for achieving a natural,intuitive,and real-time human-robotic interaction.
基金support from the National Natural Science Foundation of China(32201179)Guangdong Basic and Applied Basic Research Foundation(2020A1515110126 and 2021A1515010130)the Fundamental Research Funds for the Central Universities(N2119006 and N2224001-10)is gratefully acknowledged.
文摘Multifunctional supramolecular ultra-tough bionic e-skin with unique durability for human–machine interaction in complex scenarios still remains challenging.Herein,we develop a skininspired ultra-tough e-skin with tunable mechanical properties by a physical cross-linking salting-freezing-thawing method.The gelling agent(β-Glycerophosphate sodium:Gp)induces the aggregation and binding of PVA molecular chains and thereby toughens them(stress up to 5.79 MPa,toughness up to 13.96 MJ m^(−3)).Notably,due to molecular self-assembly,hydrogels can be fully recycled and reprocessed by direct heating(100°C for a few seconds),and the tensile strength can still be maintained at about 100%after six recoveries.The hydrogel integrates transparency(>60%),super toughness(up to 13.96 MJ m^(−3),bearing 1500 times of its own tensile weight),good antibacterial properties(E.coli and S.aureus),UV protection(Filtration:80%–90%),high electrical conductivity(4.72 S m^(−1)),anti-swelling and recyclability.The hydrogel can not only monitor daily physiological activities,but also be used for complex activities underwater and message encryption/decryption.We also used it to create a complete finger joint rehabilitation system with an interactive interface that dynamically presents the user’s health status.Our multifunctional electronic skin will have a profound impact on the future of new rehabilitation medical,human–machine interaction,VR/AR and the metaverse fields.
