Lip language provides a silent,intuitive,and efficient mode of communication,offering a promising solution for individuals with speech impairments.Its articulation relies on complex movements of the jaw and the muscle...Lip language provides a silent,intuitive,and efficient mode of communication,offering a promising solution for individuals with speech impairments.Its articulation relies on complex movements of the jaw and the muscles surrounding it.However,the accurate and real-time acquisition and decoding of these movements into reliable silent speech signals remains a significant challenge.In this work,we propose a real-time silent speech recognition system,which integrates a triboelectric nanogenerator-based flexible pressure sensor(FPS)with a deep learning framework.The FPS employs a porous pyramid-structured silicone film as the negative triboelectric layer,enabling highly sensitive pressure detection in the low-force regime(1 V N^(-1) for 0-10 N and 4.6 V N^(-1) for 10-24 N).This allows it to precisely capture jaw movements during speech and convert them into electrical signals.To decode the signals,we proposed a convolutional neural networklong short-term memory(CNN-LSTM)hybrid network,combining CNN and LSTM model to extract both local spatial features and temporal dynamics.The model achieved 95.83%classification accuracy in 30 categories of daily words.Furthermore,the decoded silent speech signals can be directly translated into executable commands for contactless and precise control of the smartphone.The system can also be connected to AR glasses,offering a novel human-machine interaction approach with promising potential in AR/VR applications.展开更多
With the rapid development of virtual reality(VR)and augmented reality(AR)technologies,their application potential in the field of education has become increasingly significant.For a long time,fire safety education in...With the rapid development of virtual reality(VR)and augmented reality(AR)technologies,their application potential in the field of education has become increasingly significant.For a long time,fire safety education in university laboratories has faced numerous challenges,and traditional teaching methods have been insufficiently effective,with high-risk scenarios difficult to realistically recreate.Especially in special scenarios involving hazardous chemicals,conventional training methods struggle to enable learners to achieve deep understanding and behavioral formation.This study systematically integrates immersive technology theory with safety education needs,providing a replicable technical solution for safety education in high-risk environments.Its modular design approach has reference value for expansion into other professional fields,offering practical evidence for innovation in safety education models in the digital age.展开更多
基金supported by the Natural Science Foundation of Fujian Province under Grant No.2024J010016Fujian Province Young and Middle aged Teacher Education Research Project No.JAT241317the Mindu Innovation Laboratory Project under Grant No.2020ZZ113.
文摘Lip language provides a silent,intuitive,and efficient mode of communication,offering a promising solution for individuals with speech impairments.Its articulation relies on complex movements of the jaw and the muscles surrounding it.However,the accurate and real-time acquisition and decoding of these movements into reliable silent speech signals remains a significant challenge.In this work,we propose a real-time silent speech recognition system,which integrates a triboelectric nanogenerator-based flexible pressure sensor(FPS)with a deep learning framework.The FPS employs a porous pyramid-structured silicone film as the negative triboelectric layer,enabling highly sensitive pressure detection in the low-force regime(1 V N^(-1) for 0-10 N and 4.6 V N^(-1) for 10-24 N).This allows it to precisely capture jaw movements during speech and convert them into electrical signals.To decode the signals,we proposed a convolutional neural networklong short-term memory(CNN-LSTM)hybrid network,combining CNN and LSTM model to extract both local spatial features and temporal dynamics.The model achieved 95.83%classification accuracy in 30 categories of daily words.Furthermore,the decoded silent speech signals can be directly translated into executable commands for contactless and precise control of the smartphone.The system can also be connected to AR glasses,offering a novel human-machine interaction approach with promising potential in AR/VR applications.
文摘With the rapid development of virtual reality(VR)and augmented reality(AR)technologies,their application potential in the field of education has become increasingly significant.For a long time,fire safety education in university laboratories has faced numerous challenges,and traditional teaching methods have been insufficiently effective,with high-risk scenarios difficult to realistically recreate.Especially in special scenarios involving hazardous chemicals,conventional training methods struggle to enable learners to achieve deep understanding and behavioral formation.This study systematically integrates immersive technology theory with safety education needs,providing a replicable technical solution for safety education in high-risk environments.Its modular design approach has reference value for expansion into other professional fields,offering practical evidence for innovation in safety education models in the digital age.
