Background Lip reading uses lip images for visual speech recognition.Deep-learning-based lip reading has greatly improved performance in current datasets;however,most existing research ignores the significance of shor...Background Lip reading uses lip images for visual speech recognition.Deep-learning-based lip reading has greatly improved performance in current datasets;however,most existing research ignores the significance of short-term temporal dependencies of lip-shape variations between adjacent frames,which leaves space for further improvement in feature extraction.Methods This article presents a spatiotemporal feature fusion network(STDNet)that compensates for the deficiencies of current lip-reading approaches in short-term temporal dependency modeling.Specifically,to distinguish more similar and intricate content,STDNet adds a temporal feature extraction branch based on a 3D-CNN,which enhances the learning of dynamic lip movements in adjacent frames while not affecting spatial feature extraction.In particular,we designed a local–temporal block,which aggregates interframe differences,strengthening the relationship between various local lip regions through multiscale convolution.We incorporated the squeeze-and-excitation mechanism into the Global-Temporal Block,which processes a single frame as an independent unitto learn temporal variations across the entire lip region more effectively.Furthermore,attention pooling was introduced to highlight meaningful frames containing key semantic information for the target word.Results Experimental results demonstrated STDNet's superior performance on the LRW and LRW-1000,achieving word-level recognition accuracies of 90.2% and 53.56%,respectively.Extensive ablation experiments verified the rationality and effectiveness of its modules.Conclusions The proposed model effectively addresses short-term temporal dependency limitations in lip reading,and improves the temporal robustness of the model against variable-length sequences.These advancements validate the importance of explicit short-term dynamics modeling for practical lip-reading systems.展开更多
Early and accurate detection of bone cancer and marrow cell abnormalities is critical for timely intervention and improved patient outcomes.This paper proposes a novel hybrid deep learning framework that integrates a ...Early and accurate detection of bone cancer and marrow cell abnormalities is critical for timely intervention and improved patient outcomes.This paper proposes a novel hybrid deep learning framework that integrates a Convolutional Neural Network(CNN)with a Bidirectional Long Short-Term Memory(BiLSTM)architecture,optimized using the Firefly Optimization algorithm(FO).The proposed CNN-BiLSTM-FO model is tailored for structured biomedical data,capturing both local patterns and sequential dependencies in diagnostic features,while the Firefly Algorithm fine-tunes key hyperparameters to maximize predictive performance.The approach is evaluated on two benchmark biomedical datasets:one comprising diagnostic data for bone cancer detection and another for identifying marrow cell abnormalities.Experimental results demonstrate that the proposed method outperforms standard deep learning models,including CNN,LSTM,BiLSTM,and CNN-LSTM hybrids,significantly.The CNNBiLSTM-FO model achieves an accuracy of 98.55%for bone cancer detection and 96.04%for marrow abnormality classification.The paper also presents a detailed complexity analysis of the proposed algorithm and compares its performance across multiple evaluation metrics such as precision,recall,F1-score,and AUC.The results confirm the effectiveness of the firefly-based optimization strategy in improving classification accuracy and model robustness.This work introduces a scalable and accurate diagnostic solution that holds strong potential for integration into intelligent clinical decision-support systems.展开更多
基金Supported by the National Key Research and Development Program of China(2023YFC3306201)the National Natural Science Foundation of China(61772125)the Fundamental Research Funds for the Central Universities(N2317004).
文摘Background Lip reading uses lip images for visual speech recognition.Deep-learning-based lip reading has greatly improved performance in current datasets;however,most existing research ignores the significance of short-term temporal dependencies of lip-shape variations between adjacent frames,which leaves space for further improvement in feature extraction.Methods This article presents a spatiotemporal feature fusion network(STDNet)that compensates for the deficiencies of current lip-reading approaches in short-term temporal dependency modeling.Specifically,to distinguish more similar and intricate content,STDNet adds a temporal feature extraction branch based on a 3D-CNN,which enhances the learning of dynamic lip movements in adjacent frames while not affecting spatial feature extraction.In particular,we designed a local–temporal block,which aggregates interframe differences,strengthening the relationship between various local lip regions through multiscale convolution.We incorporated the squeeze-and-excitation mechanism into the Global-Temporal Block,which processes a single frame as an independent unitto learn temporal variations across the entire lip region more effectively.Furthermore,attention pooling was introduced to highlight meaningful frames containing key semantic information for the target word.Results Experimental results demonstrated STDNet's superior performance on the LRW and LRW-1000,achieving word-level recognition accuracies of 90.2% and 53.56%,respectively.Extensive ablation experiments verified the rationality and effectiveness of its modules.Conclusions The proposed model effectively addresses short-term temporal dependency limitations in lip reading,and improves the temporal robustness of the model against variable-length sequences.These advancements validate the importance of explicit short-term dynamics modeling for practical lip-reading systems.
文摘Early and accurate detection of bone cancer and marrow cell abnormalities is critical for timely intervention and improved patient outcomes.This paper proposes a novel hybrid deep learning framework that integrates a Convolutional Neural Network(CNN)with a Bidirectional Long Short-Term Memory(BiLSTM)architecture,optimized using the Firefly Optimization algorithm(FO).The proposed CNN-BiLSTM-FO model is tailored for structured biomedical data,capturing both local patterns and sequential dependencies in diagnostic features,while the Firefly Algorithm fine-tunes key hyperparameters to maximize predictive performance.The approach is evaluated on two benchmark biomedical datasets:one comprising diagnostic data for bone cancer detection and another for identifying marrow cell abnormalities.Experimental results demonstrate that the proposed method outperforms standard deep learning models,including CNN,LSTM,BiLSTM,and CNN-LSTM hybrids,significantly.The CNNBiLSTM-FO model achieves an accuracy of 98.55%for bone cancer detection and 96.04%for marrow abnormality classification.The paper also presents a detailed complexity analysis of the proposed algorithm and compares its performance across multiple evaluation metrics such as precision,recall,F1-score,and AUC.The results confirm the effectiveness of the firefly-based optimization strategy in improving classification accuracy and model robustness.This work introduces a scalable and accurate diagnostic solution that holds strong potential for integration into intelligent clinical decision-support systems.
