The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extrac...The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extract knowledge from these sources is imperative.Recently,the BlazePose system has been released for skeleton extraction from images oriented to mobile devices.With this skeleton graph representation in place,a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action.We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest,it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks.Hence,in this study,we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition.Moreover,we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor.Additionally,we propose different skeleton detection thresholds that can improve the accuracy performance even further.We reached a top-1 accuracy performance of 40.1%on the Kinetics dataset.For the NTU-RGB+D dataset,we achieved 87.59%and 92.1%accuracy for Cross-Subject and Cross-View evaluation criteria,respectively.展开更多
A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain...A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain the spatial structure information of human motion and extract the correlation in the time series of human motion.The residual structure is applied to the proposed network model to alleviate the problem of gradient disappearance in the deep network.Experiments on the Human 3.6M dataset demonstrate that the proposed method effectively reduces the errors of motion prediction compared with previous methods,especially of long-term prediction.展开更多
Diabetes,as a chronic disease,is caused by the increase of blood glucose concentration due to pancreatic insulin production failure or insulin resistance in the body.Predicting the change trend of blood glucose level ...Diabetes,as a chronic disease,is caused by the increase of blood glucose concentration due to pancreatic insulin production failure or insulin resistance in the body.Predicting the change trend of blood glucose level in advance brings convenience for prompt treatment,so as to maintain blood glucose level within the recommended levels.Based on the flash glucose monitoring data,we propose a method that combines prophet with temporal convolutional networks(TCN)to achieve good experimental results in predicting patient blood glucose.The proposed model achieves high accuracy in the long-term and short-term prediction of blood glucose,and outperforms other models on the adaptability to non-stationary and detection capability of periodic changes.展开更多
Traffic flow forecasting plays a crucial role and is the key technology to realize dynamic traffic guidance and active traffic control in intelligent traffic systems(ITS).Aiming at the complex local and global spatial...Traffic flow forecasting plays a crucial role and is the key technology to realize dynamic traffic guidance and active traffic control in intelligent traffic systems(ITS).Aiming at the complex local and global spatial-temporal dynamic characteristics of traffic flow,this paper proposes a new traffic flow forecasting model spatial-temporal attention graph neural network(STA-GNN)by combining at-tention mechanism(AM)and spatial-temporal convolutional network.The model learns the hidden dynamic local spatial correlations of the traffic network by combining the dynamic adjacency matrix constructed by the graph learning layer with the graph convolutional network(GCN).The local tem-poral correlations of traffic flow at different scales are extracted by stacking multiple convolutional kernels in temporal convolutional network(TCN).And the global spatial-temporal dependencies of long-time sequences of traffic flow are captured by the spatial-temporal attention mechanism(STAtt),which enhances the global spatial-temporal modeling and the representational ability of model.The experimental results on two datasets,METR-LA and PEMS-BAY,show the proposed STA-GNN model outperforms the common baseline models in forecasting accuracy.展开更多
Generating realistic and synthetic video from text is a highly challenging task due to the multitude of issues involved,including digit deformation,noise interference between frames,blurred output,and the need for tem...Generating realistic and synthetic video from text is a highly challenging task due to the multitude of issues involved,including digit deformation,noise interference between frames,blurred output,and the need for temporal coherence across frames.In this paper,we propose a novel approach for generating coherent videos of moving digits from textual input using a Deep Deconvolutional Generative Adversarial Network(DD-GAN).The DDGAN comprises a Deep Deconvolutional Neural Network(DDNN)as a Generator(G)and a modified Deep Convolutional Neural Network(DCNN)as a Discriminator(D)to ensure temporal coherence between adjacent frames.The proposed research involves several steps.First,the input text is fed into a Long Short Term Memory(LSTM)based text encoder and then smoothed using Conditioning Augmentation(CA)techniques to enhance the effectiveness of the Generator(G).Next,using a DDNN to generate video frames by incorporating enhanced text and random noise and modifying a DCNN to act as a Discriminator(D),effectively distinguishing between generated and real videos.This research evaluates the quality of the generated videos using standard metrics like Inception Score(IS),Fréchet Inception Distance(FID),Fréchet Inception Distance for video(FID2vid),and Generative Adversarial Metric(GAM),along with a human study based on realism,coherence,and relevance.By conducting experiments on Single-Digit Bouncing MNIST GIFs(SBMG),Two-Digit Bouncing MNIST GIFs(TBMG),and a custom dataset of essential mathematics videos with related text,this research demonstrates significant improvements in both metrics and human study results,confirming the effectiveness of DD-GAN.This research also took the exciting challenge of generating preschool math videos from text,handling complex structures,digits,and symbols,and achieving successful results.The proposed research demonstrates promising results for generating coherent videos from textual input.展开更多
文摘The ever-growing available visual data(i.e.,uploaded videos and pictures by internet users)has attracted the research community’s attention in the computer vision field.Therefore,finding efficient solutions to extract knowledge from these sources is imperative.Recently,the BlazePose system has been released for skeleton extraction from images oriented to mobile devices.With this skeleton graph representation in place,a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action.We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest,it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks.Hence,in this study,we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition.Moreover,we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor.Additionally,we propose different skeleton detection thresholds that can improve the accuracy performance even further.We reached a top-1 accuracy performance of 40.1%on the Kinetics dataset.For the NTU-RGB+D dataset,we achieved 87.59%and 92.1%accuracy for Cross-Subject and Cross-View evaluation criteria,respectively.
文摘A lightweight multi-layer residual temporal convolutional network model(RTCN)is proposed to target the highly complex kinematics and temporal correlation of human motion.RTCN uses 1-D convolution to efficiently obtain the spatial structure information of human motion and extract the correlation in the time series of human motion.The residual structure is applied to the proposed network model to alleviate the problem of gradient disappearance in the deep network.Experiments on the Human 3.6M dataset demonstrate that the proposed method effectively reduces the errors of motion prediction compared with previous methods,especially of long-term prediction.
文摘Diabetes,as a chronic disease,is caused by the increase of blood glucose concentration due to pancreatic insulin production failure or insulin resistance in the body.Predicting the change trend of blood glucose level in advance brings convenience for prompt treatment,so as to maintain blood glucose level within the recommended levels.Based on the flash glucose monitoring data,we propose a method that combines prophet with temporal convolutional networks(TCN)to achieve good experimental results in predicting patient blood glucose.The proposed model achieves high accuracy in the long-term and short-term prediction of blood glucose,and outperforms other models on the adaptability to non-stationary and detection capability of periodic changes.
基金Supported by the Key R&D Program of Gansu Province(No.23YFGA0063)the National Natural Science Foundation of China(No.62363022,61663021)+1 种基金the Natural Science Foundation of Gansu Province(No.22JR5RA226,23JRRA886)the Gansu Provincial De-partment of Education:Industrial Support Plan Project(No.2023CYZC-35).
文摘Traffic flow forecasting plays a crucial role and is the key technology to realize dynamic traffic guidance and active traffic control in intelligent traffic systems(ITS).Aiming at the complex local and global spatial-temporal dynamic characteristics of traffic flow,this paper proposes a new traffic flow forecasting model spatial-temporal attention graph neural network(STA-GNN)by combining at-tention mechanism(AM)and spatial-temporal convolutional network.The model learns the hidden dynamic local spatial correlations of the traffic network by combining the dynamic adjacency matrix constructed by the graph learning layer with the graph convolutional network(GCN).The local tem-poral correlations of traffic flow at different scales are extracted by stacking multiple convolutional kernels in temporal convolutional network(TCN).And the global spatial-temporal dependencies of long-time sequences of traffic flow are captured by the spatial-temporal attention mechanism(STAtt),which enhances the global spatial-temporal modeling and the representational ability of model.The experimental results on two datasets,METR-LA and PEMS-BAY,show the proposed STA-GNN model outperforms the common baseline models in forecasting accuracy.
基金supported by the General Program of the National Natural Science Foundation of China(Grant No.61977029).
文摘Generating realistic and synthetic video from text is a highly challenging task due to the multitude of issues involved,including digit deformation,noise interference between frames,blurred output,and the need for temporal coherence across frames.In this paper,we propose a novel approach for generating coherent videos of moving digits from textual input using a Deep Deconvolutional Generative Adversarial Network(DD-GAN).The DDGAN comprises a Deep Deconvolutional Neural Network(DDNN)as a Generator(G)and a modified Deep Convolutional Neural Network(DCNN)as a Discriminator(D)to ensure temporal coherence between adjacent frames.The proposed research involves several steps.First,the input text is fed into a Long Short Term Memory(LSTM)based text encoder and then smoothed using Conditioning Augmentation(CA)techniques to enhance the effectiveness of the Generator(G).Next,using a DDNN to generate video frames by incorporating enhanced text and random noise and modifying a DCNN to act as a Discriminator(D),effectively distinguishing between generated and real videos.This research evaluates the quality of the generated videos using standard metrics like Inception Score(IS),Fréchet Inception Distance(FID),Fréchet Inception Distance for video(FID2vid),and Generative Adversarial Metric(GAM),along with a human study based on realism,coherence,and relevance.By conducting experiments on Single-Digit Bouncing MNIST GIFs(SBMG),Two-Digit Bouncing MNIST GIFs(TBMG),and a custom dataset of essential mathematics videos with related text,this research demonstrates significant improvements in both metrics and human study results,confirming the effectiveness of DD-GAN.This research also took the exciting challenge of generating preschool math videos from text,handling complex structures,digits,and symbols,and achieving successful results.The proposed research demonstrates promising results for generating coherent videos from textual input.
