Technique for horror video recognition is important for its application in web content filtering and surveillance, especially for preventing children from being threaten. In this paper, a novel horror video recognitio...Technique for horror video recognition is important for its application in web content filtering and surveillance, especially for preventing children from being threaten. In this paper, a novel horror video recognition algorithm based on fuzzy comprehensive evolution model is proposed. Three low-level video features are extracted as typical features, and they are video key-light, video colour energy and video rhythm. Analytic Hierarchy Process （AHP） is adopted to estimate the weights of extracted features in fuzzy evolution model. Horror evaluation （membership function） is on shot scale and it is constructed based on the knowledge that videos which share the same affective have similar low-level features. K-Means algorithm is implemented to help finding the most representative feature vectors. The experimental results demonstrate that the proposed approach has good performance in recognition precision, recall rate and F1 measure.展开更多
Vehicle–bicycle conflict incurs a higher risk of traffic accidents,particularly as it frequently takes place at intersections.Mastering the traffic characteristics of vehicle–bicycle conflict and optimizing the desi...Vehicle–bicycle conflict incurs a higher risk of traffic accidents,particularly as it frequently takes place at intersections.Mastering the traffic characteristics of vehicle–bicycle conflict and optimizing the design of intersections can effectively reduce such conflict.In this paper,the conflict between right-turning motor vehicles and straight-riding bicycles was taken as the research object,and T-Analyst video recognition technology was used to obtain data on riding(driving)behavior and vehicle–bicycle conflict at seven intersections in Changsha,China.Herein,eight typical traffic characteristics of vehicle–bicycle conflict are summarized,the causes of vehicle–bicycle conflict are analyzed using 18 factors in three dimensions,the internal relationship between intersection design factors and traffic conflicts is explored,and the guiding of design optimization based on the width of bicycle lanes and the soft separation between vehicles and bicycles is discussed.The results showed that colored paved bicycle lanes were better,performing better at a width of 2.5 m compared to 1.5 m.However,the colored pavement was not suitable for the entire road and had to be set at the position,at which the trajectories of a bicycle and motor vehicle overlapped.Thus,a 2.5-m-wide bicycle lane provides good safety.However,there are still defects in the existing safety indicators,so it is necessary to develop new indicators to reflect real vehicle–bicycle conflict situations more comprehensively.展开更多
To study riding safety at intersection entrance,video recognition technology is used to build vehicle-bicycle conflict models based on the Bayesian method.It is analyzed the relationship among the width of nonmotorize...To study riding safety at intersection entrance,video recognition technology is used to build vehicle-bicycle conflict models based on the Bayesian method.It is analyzed the relationship among the width of nonmotorized lanes at the entrance lane of the intersection,the vehicle-bicycle soft isolation form of the entrance lane of intersection,the traffic volume of right-turning motor vehicles and straight-going non-motor vehicles,the speed of right-turning motor vehicles,and straight-going non-motor vehicles,and the conflict between right-turning motor vehicles and straight-going nonmotor vehicles.Due to the traditional statistical methods,to overcome the discreteness of vehicle-bicycle conflict data and the differences of influencing factors,the Bayesian random effect Poisson-log-normal model and random effect negative binomial regression model are established.The results show that the random effect Poisson-log-normal model is better than the negative binomial distribution of random effects;The width of non-motorized lanes,the form of vehicle-bicycle soft isolation,the traffic volume of right-turning motor vehicles,and the coefficients of straight traffic volume obey a normal distribution.Among them,the type of vehicle-bicycle soft isolation facilities and the vehicle-bicycle traffic volumes are significantly positively correlated with the number of vehicle-bicycle conflicts.The width of non-motorized lanes is significantly negatively correlated with the number of vehicle-bicycle conflicts.Peak periods and flat periods,the average speed of right-turning motor vehicles,and the average speed of straight-going non-motor vehicles have no significant influence on the number of vehicle-bicycle conflicts.展开更多
Video action recognition(VAR)aims to analyze dynamic behaviors in videos and achieve semantic understanding.VAR faces challenges such as temporal dynamics,action-scene coupling,and the complexity of human interactions...Video action recognition(VAR)aims to analyze dynamic behaviors in videos and achieve semantic understanding.VAR faces challenges such as temporal dynamics,action-scene coupling,and the complexity of human interactions.Existing methods can be categorized into motion-level,event-level,and story-level ones based on spatiotemporal granularity.However,single-modal approaches struggle to capture complex behavioral semantics and human factors.Therefore,in recent years,vision-language models(VLMs)have been introduced into this field,providing new research perspectives for VAR.In this paper,we systematically review spatiotemporal hierarchical methods in VAR and explore how the introduction of large models has advanced the field.Additionally,we propose the concept of“Factor”to identify and integrate key information from both visual and textual modalities,enhancing multimodal alignment.We also summarize various multimodal alignment methods and provide in-depth analysis and insights into future research directions.展开更多
With the rapid development of artificial intelligence and Internet of Things technologies,video action recognition technology is widely applied in various scenarios,such as personal life and industrial production.Howe...With the rapid development of artificial intelligence and Internet of Things technologies,video action recognition technology is widely applied in various scenarios,such as personal life and industrial production.However,while enjoying the convenience brought by this technology,it is crucial to effectively protect the privacy of users’video data.Therefore,this paper proposes a video action recognition method based on personalized federated learning and spatiotemporal features.Under the framework of federated learning,a video action recognition method leveraging spatiotemporal features is designed.For the local spatiotemporal features of the video,a new differential information extraction scheme is proposed to extract differential features with a single RGB frame as the center,and a spatialtemporal module based on local information is designed to improve the effectiveness of local feature extraction;for the global temporal features,a method of extracting action rhythm features using differential technology is proposed,and a timemodule based on global information is designed.Different translational strides are used in the module to obtain bidirectional differential features under different action rhythms.Additionally,to address user data privacy issues,the method divides model parameters into local private parameters and public parameters based on the structure of the video action recognition model.This approach enhancesmodel training performance and ensures the security of video data.The experimental results show that under personalized federated learning conditions,an average accuracy of 97.792%was achieved on the UCF-101 dataset,which is non-independent and identically distributed(non-IID).This research provides technical support for privacy protection in video action recognition.展开更多
Pomo video recognition is important for Intemet content monitoring. In this paper, a novel pomo video recognition method by fusing the audio and video cues is proposed. Firstly, global color and texture features and l...Pomo video recognition is important for Intemet content monitoring. In this paper, a novel pomo video recognition method by fusing the audio and video cues is proposed. Firstly, global color and texture features and local scale-invariant feature transform （SIFT） are extracted to train multiple support vector machine （SVM） classifiers for different erotic categories of image frames. And then, two continuous density hidden Markov models （CHMM） are built to recognize porno sounds. Finally, a fusion method based on Bayes rule is employed to combine the classification results by video and audio cues. The experimental results show that our model is better than six state-of-the-art methods.展开更多
In the current era of multimedia information,it is increasingly urgent to realize intelligent video action recognition and content analysis.In the past few years,video action recognition,as an important direction in c...In the current era of multimedia information,it is increasingly urgent to realize intelligent video action recognition and content analysis.In the past few years,video action recognition,as an important direction in computer vision,has attracted many researchers and made much progress.First,this paper reviews the latest video action recognition methods based on Deep Neural Network and Markov Logic Network.Second,we analyze the characteristics of each method and the performance from the experiment results.Then compare the emphases of these methods and discuss the application scenarios.Finally,we consider and prospect the development trend and direction of this field.展开更多
Video events recognition is a challenging task for high-level understanding of video se- quence. At present, there are two major limitations in existing methods for events recognition. One is that no algorithms are av...Video events recognition is a challenging task for high-level understanding of video se- quence. At present, there are two major limitations in existing methods for events recognition. One is that no algorithms are available to recognize events which happen alternately. The other is that the temporal relationship between atomic actions is not fully utilized. Aiming at these problems, an algo- rithm based on an extended stochastic context-free grammar （SCFG） representation is proposed for events recognition. Events are modeled by a series of atomic actions and represented by an extended SCFG. The extended SCFG can express the hierarchical structure of the events and the temporal re- lationship between the atomic actions. In comparison with previous work, the main contributions of this paper are as follows： ① Events （include alternating events） can be recognized by an improved stochastic parsing and shortest path finding algorithm. ② The algorithm can disambiguate the detec- tion results of atomic actions by event context. Experimental results show that the proposed algo- rithm can recognize events accurately and most atomic action detection errors can be corrected sim- ultaneously.展开更多
In this paper, we propose a video searching system that utilizes face recognition as searching indexing feature. As the applications of video cameras have great increase in recent years, face recognition makes a perfe...In this paper, we propose a video searching system that utilizes face recognition as searching indexing feature. As the applications of video cameras have great increase in recent years, face recognition makes a perfect fit for searching targeted individuals within the vast amount of video data. However, the performance of such searching depends on the quality of face images recorded in the video signals. Since the surveillance video cameras record videos without fixed postures for the object, face occlusion is very common in everyday video. The proposed system builds a model for occluded faces using fuzzy principal component analysis(FPCA), and reconstructs the human faces with the available information. Experimental results show that the system has very high efficiency in processing the real life videos, and it is very robust to various kinds of face occlusions. Hence it can relieve people reviewers from the front of the monitors and greatly enhances the efficiency as well. The proposed system has been installed and applied in various environments and has already demonstrated its power by helping solving real cases.展开更多
An exhaustive study has been conducted on face videos from YouTube video dataset for real time face recognition using the features from deep learning architectures and also the information set features. Our objective ...An exhaustive study has been conducted on face videos from YouTube video dataset for real time face recognition using the features from deep learning architectures and also the information set features. Our objective is to cash in on a plethora of deep learning architectures and information set features. The deep learning architectures dig in features from several layers of convolution and max-pooling layers though a placement of these layers is architecture dependent. On the other hand, the information set features depend on the entropy function for the generation of features. A comparative study of deep learning and information set features is made using the well-known classifiers in addition to developing Constrained Hanman Transform (CHT) and Weighted Hanman Transform (WHT) classifiers. It is demonstrated that information set features and deep learning features have comparable performance. However, sigmoid-based information set features using the new classifiers are found to outperform MobileNet features.展开更多
Although recurrent neural networks(RNNs)are widely leveraged to process temporal or sequential data,they have attracted too little attention in current video action recognition applications.Therefore,this work attempt...Although recurrent neural networks(RNNs)are widely leveraged to process temporal or sequential data,they have attracted too little attention in current video action recognition applications.Therefore,this work attempts to model the long-term spatio-temporal information of the video based on a variant of RNN,i.e.,higher-order RNN.Moreover,we propose a novel long-term spatio-temporal network(LSN)for solving this video task,the core of which integrates the newly constructed high-order ConvLSTM(HO-ConvLSTM)modules with traditional 2D convolutional blocks.Specifically,each HO-ConvLSTM module consists of an accumulated temporary state(ATS)module as well as a standard ConvLSTM module,and several previous hidden states in the ATS module are accumulated to one temporary state that will enter the standard ConvLSTM to determine the output together with the current input.The HO-ConvLSTM module can be inserted into different stages of the 2D convolutional neural network(CNN)in a plug-andplay manner,thus well characterizing the long-term temporal evolution at various spatial resolutions.Experiment results on three commonly used video benchmarks demonstrate that the proposed LSN model can achieve competitive performance with the representative models.展开更多
文摘Technique for horror video recognition is important for its application in web content filtering and surveillance, especially for preventing children from being threaten. In this paper, a novel horror video recognition algorithm based on fuzzy comprehensive evolution model is proposed. Three low-level video features are extracted as typical features, and they are video key-light, video colour energy and video rhythm. Analytic Hierarchy Process （AHP） is adopted to estimate the weights of extracted features in fuzzy evolution model. Horror evaluation （membership function） is on shot scale and it is constructed based on the knowledge that videos which share the same affective have similar low-level features. K-Means algorithm is implemented to help finding the most representative feature vectors. The experimental results demonstrate that the proposed approach has good performance in recognition precision, recall rate and F1 measure.
基金This work was supported in part by the Ministry of Education of the People’s Republic of China Project of Humanities and Social Sciences under Grant No.19YJCZH208,author X.X,http://www.moe.gov.cn/in part by the Philosophy and Social Science Foundation Project of Hunan Province under Grant No.15YBA406,author X.X,http://www.hnsk.gov.cn/+3 种基金in part by the Social Science Evaluation Committee Project of Hunan Province under Grant No.XSP18YBZ125,author X.X,http://www.hnsk.gov.cn/in part by the Social Sciences Federation Think Tank Project of Hunan Province under Grant No.ZK2019025,author X.X,http://www.hnsk.gov.cn/in part by the Education Bureau Research Foundation Project of Hunan Province under Grant No.20A531,author X.X,http://jyt.hunan.gov.cn/in part by the Science and Technology Project of Changsha City,under Grant No.kq2004092,author X.X,http://kjj.changsha.gov.cn/.
文摘Vehicle–bicycle conflict incurs a higher risk of traffic accidents,particularly as it frequently takes place at intersections.Mastering the traffic characteristics of vehicle–bicycle conflict and optimizing the design of intersections can effectively reduce such conflict.In this paper,the conflict between right-turning motor vehicles and straight-riding bicycles was taken as the research object,and T-Analyst video recognition technology was used to obtain data on riding(driving)behavior and vehicle–bicycle conflict at seven intersections in Changsha,China.Herein,eight typical traffic characteristics of vehicle–bicycle conflict are summarized,the causes of vehicle–bicycle conflict are analyzed using 18 factors in three dimensions,the internal relationship between intersection design factors and traffic conflicts is explored,and the guiding of design optimization based on the width of bicycle lanes and the soft separation between vehicles and bicycles is discussed.The results showed that colored paved bicycle lanes were better,performing better at a width of 2.5 m compared to 1.5 m.However,the colored pavement was not suitable for the entire road and had to be set at the position,at which the trajectories of a bicycle and motor vehicle overlapped.Thus,a 2.5-m-wide bicycle lane provides good safety.However,there are still defects in the existing safety indicators,so it is necessary to develop new indicators to reflect real vehicle–bicycle conflict situations more comprehensively.
基金This work was supported in part by the Ministry of Education of the People’s Republic of China Project of Humanities and Social Sciences under Grant No.19YJCZH208,author X.X,http://www.moe.gov.cn/in part by the Social Sciences Federation Think Tank Project of Hunan Province under Grant No.ZK2019025,author X.X,http://www.hnsk.gov.cn/+3 种基金in part by the Education Bureau Research Foundation Project of Hunan Province under Grant No.20A531,author X.X,http://jyt.hunan.gov.cn/in part by the Science and Technology Project of Changsha City,under Grant No.kq2004092,author X.X,http://kjj.changsha.gov.cn/in part by Key Subjects of the State Forestry Bureau in China under Grant No.[2016]21,author X.X,http://www.forestry.gov.cn/and in part by“Double First-Class”Cultivation Discipline of Hunan Province in China under Grant No.[2018]469,author X.X,http://jyt.hunan.gov.cn/.
文摘To study riding safety at intersection entrance,video recognition technology is used to build vehicle-bicycle conflict models based on the Bayesian method.It is analyzed the relationship among the width of nonmotorized lanes at the entrance lane of the intersection,the vehicle-bicycle soft isolation form of the entrance lane of intersection,the traffic volume of right-turning motor vehicles and straight-going non-motor vehicles,the speed of right-turning motor vehicles,and straight-going non-motor vehicles,and the conflict between right-turning motor vehicles and straight-going nonmotor vehicles.Due to the traditional statistical methods,to overcome the discreteness of vehicle-bicycle conflict data and the differences of influencing factors,the Bayesian random effect Poisson-log-normal model and random effect negative binomial regression model are established.The results show that the random effect Poisson-log-normal model is better than the negative binomial distribution of random effects;The width of non-motorized lanes,the form of vehicle-bicycle soft isolation,the traffic volume of right-turning motor vehicles,and the coefficients of straight traffic volume obey a normal distribution.Among them,the type of vehicle-bicycle soft isolation facilities and the vehicle-bicycle traffic volumes are significantly positively correlated with the number of vehicle-bicycle conflicts.The width of non-motorized lanes is significantly negatively correlated with the number of vehicle-bicycle conflicts.Peak periods and flat periods,the average speed of right-turning motor vehicles,and the average speed of straight-going non-motor vehicles have no significant influence on the number of vehicle-bicycle conflicts.
基金supported by the Zhejiang Provincial Natural Science Foundation of China(No.LQ23F030001)the National Natural Science Foundation of China(No.62406280)+5 种基金the Autism Research Special Fund of Zhejiang Foundation for Disabled Persons(No.2023008)the Liaoning Province Higher Education Innovative Talents Program Support Project(No.LR2019058)the Liaoning Province Joint Open Fund for Key Scientific and Technological Innovation Bases(No.2021-KF-12-05)the Central Guidance on Local Science and Technology Development Fund of Liaoning Province(No.2023JH6/100100066)the Key Laboratory for Biomedical Engineering of Ministry of Education,Zhejiang University,Chinain part by the Open Research Fund of the State Key Laboratory of Cognitive Neuroscience and Learning.
文摘Video action recognition(VAR)aims to analyze dynamic behaviors in videos and achieve semantic understanding.VAR faces challenges such as temporal dynamics,action-scene coupling,and the complexity of human interactions.Existing methods can be categorized into motion-level,event-level,and story-level ones based on spatiotemporal granularity.However,single-modal approaches struggle to capture complex behavioral semantics and human factors.Therefore,in recent years,vision-language models(VLMs)have been introduced into this field,providing new research perspectives for VAR.In this paper,we systematically review spatiotemporal hierarchical methods in VAR and explore how the introduction of large models has advanced the field.Additionally,we propose the concept of“Factor”to identify and integrate key information from both visual and textual modalities,enhancing multimodal alignment.We also summarize various multimodal alignment methods and provide in-depth analysis and insights into future research directions.
基金supported by National Natural Science Foundation of China(Grant No.62071098)Sichuan Science and Technology Program(Grants 2022YFG0319,2023YFG0301 and 2023YFG0018).
文摘With the rapid development of artificial intelligence and Internet of Things technologies,video action recognition technology is widely applied in various scenarios,such as personal life and industrial production.However,while enjoying the convenience brought by this technology,it is crucial to effectively protect the privacy of users’video data.Therefore,this paper proposes a video action recognition method based on personalized federated learning and spatiotemporal features.Under the framework of federated learning,a video action recognition method leveraging spatiotemporal features is designed.For the local spatiotemporal features of the video,a new differential information extraction scheme is proposed to extract differential features with a single RGB frame as the center,and a spatialtemporal module based on local information is designed to improve the effectiveness of local feature extraction;for the global temporal features,a method of extracting action rhythm features using differential technology is proposed,and a timemodule based on global information is designed.Different translational strides are used in the module to obtain bidirectional differential features under different action rhythms.Additionally,to address user data privacy issues,the method divides model parameters into local private parameters and public parameters based on the structure of the video action recognition model.This approach enhancesmodel training performance and ensures the security of video data.The experimental results show that under personalized federated learning conditions,an average accuracy of 97.792%was achieved on the UCF-101 dataset,which is non-independent and identically distributed(non-IID).This research provides technical support for privacy protection in video action recognition.
基金supported by the National Natural Science Foundation of China (90920001, 61101212)the Fundamental Research Funds for the Central Universities
文摘Pomo video recognition is important for Intemet content monitoring. In this paper, a novel pomo video recognition method by fusing the audio and video cues is proposed. Firstly, global color and texture features and local scale-invariant feature transform （SIFT） are extracted to train multiple support vector machine （SVM） classifiers for different erotic categories of image frames. And then, two continuous density hidden Markov models （CHMM） are built to recognize porno sounds. Finally, a fusion method based on Bayes rule is employed to combine the classification results by video and audio cues. The experimental results show that our model is better than six state-of-the-art methods.
基金This work was supported in part by National Science Foundation Project of P.R.China(Grant Nos.61503424,61331013)。
文摘In the current era of multimedia information,it is increasingly urgent to realize intelligent video action recognition and content analysis.In the past few years,video action recognition,as an important direction in computer vision,has attracted many researchers and made much progress.First,this paper reviews the latest video action recognition methods based on Deep Neural Network and Markov Logic Network.Second,we analyze the characteristics of each method and the performance from the experiment results.Then compare the emphases of these methods and discuss the application scenarios.Finally,we consider and prospect the development trend and direction of this field.
基金Supported by the National Natural Science Foundation of China(60805028,60903146)Natural Science Foundation of Shandong Province of China (ZR2010FM027)+1 种基金SDUST Research Fund(2010KYTD101)China Postdoctoral Science Foundation(2012M521336)
文摘Video events recognition is a challenging task for high-level understanding of video se- quence. At present, there are two major limitations in existing methods for events recognition. One is that no algorithms are available to recognize events which happen alternately. The other is that the temporal relationship between atomic actions is not fully utilized. Aiming at these problems, an algo- rithm based on an extended stochastic context-free grammar （SCFG） representation is proposed for events recognition. Events are modeled by a series of atomic actions and represented by an extended SCFG. The extended SCFG can express the hierarchical structure of the events and the temporal re- lationship between the atomic actions. In comparison with previous work, the main contributions of this paper are as follows： ① Events （include alternating events） can be recognized by an improved stochastic parsing and shortest path finding algorithm. ② The algorithm can disambiguate the detec- tion results of atomic actions by event context. Experimental results show that the proposed algo- rithm can recognize events accurately and most atomic action detection errors can be corrected sim- ultaneously.
基金supported by the National Natural Science Foundation of China(No.61502256)
文摘In this paper, we propose a video searching system that utilizes face recognition as searching indexing feature. As the applications of video cameras have great increase in recent years, face recognition makes a perfect fit for searching targeted individuals within the vast amount of video data. However, the performance of such searching depends on the quality of face images recorded in the video signals. Since the surveillance video cameras record videos without fixed postures for the object, face occlusion is very common in everyday video. The proposed system builds a model for occluded faces using fuzzy principal component analysis(FPCA), and reconstructs the human faces with the available information. Experimental results show that the system has very high efficiency in processing the real life videos, and it is very robust to various kinds of face occlusions. Hence it can relieve people reviewers from the front of the monitors and greatly enhances the efficiency as well. The proposed system has been installed and applied in various environments and has already demonstrated its power by helping solving real cases.
文摘An exhaustive study has been conducted on face videos from YouTube video dataset for real time face recognition using the features from deep learning architectures and also the information set features. Our objective is to cash in on a plethora of deep learning architectures and information set features. The deep learning architectures dig in features from several layers of convolution and max-pooling layers though a placement of these layers is architecture dependent. On the other hand, the information set features depend on the entropy function for the generation of features. A comparative study of deep learning and information set features is made using the well-known classifiers in addition to developing Constrained Hanman Transform (CHT) and Weighted Hanman Transform (WHT) classifiers. It is demonstrated that information set features and deep learning features have comparable performance. However, sigmoid-based information set features using the new classifiers are found to outperform MobileNet features.
基金supported by the National Natural Science Foundation of China (61972062,61902220)the Young and Middle-aged Talents Program of the National Civil Affairs Commission,and the University-Industry Collaborative Education Program (201902029013).
文摘Although recurrent neural networks(RNNs)are widely leveraged to process temporal or sequential data,they have attracted too little attention in current video action recognition applications.Therefore,this work attempts to model the long-term spatio-temporal information of the video based on a variant of RNN,i.e.,higher-order RNN.Moreover,we propose a novel long-term spatio-temporal network(LSN)for solving this video task,the core of which integrates the newly constructed high-order ConvLSTM(HO-ConvLSTM)modules with traditional 2D convolutional blocks.Specifically,each HO-ConvLSTM module consists of an accumulated temporary state(ATS)module as well as a standard ConvLSTM module,and several previous hidden states in the ATS module are accumulated to one temporary state that will enter the standard ConvLSTM to determine the output together with the current input.The HO-ConvLSTM module can be inserted into different stages of the 2D convolutional neural network(CNN)in a plug-andplay manner,thus well characterizing the long-term temporal evolution at various spatial resolutions.Experiment results on three commonly used video benchmarks demonstrate that the proposed LSN model can achieve competitive performance with the representative models.
