Video prediction is the problem of generating future frames by exploiting the spatiotemporal correlation from the past frame sequence.It is one of the crucial issues in computer vision and has many real-world applicat...Video prediction is the problem of generating future frames by exploiting the spatiotemporal correlation from the past frame sequence.It is one of the crucial issues in computer vision and has many real-world applications,mainly focused on predicting future scenarios to avoid undesirable outcomes.However,modeling future image content and object is challenging due to the dynamic evolution and complexity of the scene,such as occlusions,camera movements,delay and illumination.Direct frame synthesis or optical-flow estimation are common approaches used by researchers.However,researchers mainly focused on video prediction using one of the approaches.Both methods have limitations,such as direct frame synthesis,usually face blurry prediction due to complex pixel distributions in the scene,and optical-flow estimation,usually produce artifacts due to large object displacements or obstructions in the clip.In this paper,we constructed a deep neural network Frame Prediction Network(FPNet-OF)with multiplebranch inputs(optical flow and original frame)to predict the future video frame by adaptively fusing the future object-motion with the future frame generator.The key idea is to jointly optimize direct RGB frame synthesis and dense optical flow estimation to generate a superior video prediction network.Using various real-world datasets,we experimentally verify that our proposed framework can produce high-level video frame compared to other state-ofthe-art framework.展开更多
Video-based vibration measurement is a cost-effective way for remote monitoring the health conditions of transportation and other civil structures, especially for situations where accessibility is restricted and does ...Video-based vibration measurement is a cost-effective way for remote monitoring the health conditions of transportation and other civil structures, especially for situations where accessibility is restricted and does not allow installation of conventional monitoring devices. Besides, video-based system is global measurement. The technical basis of video-based remote vibration measurement system is digital image analysis. Comparison of the images allow the field of motion to be accurately delineated. Such information is important to understand the structure behaviors including the motion and strain distribution. This paper presents system and analyses to monitor the vibration velocity and displacement field. The performance is demonstrated on a testbed of model building. Three different methods （i.e., frame difference method, particle image velocimetry, and optical Flow Method） are utilized to analyze the image sequences to extract the feature of motion. The Performance is validated using accelerometer data. The results indicate that all three methods can estimate the velocity field of the model building, although the results can be affected by factors such as background noise and environmental interference. Optical flow method achieved the best performance among these three methods studied. With further refinement of system hardware and image processing software, it will be developed into a remote video based monitoring system for structural health monitoring of transportation infrastructure to assist the diagnoses of its health conditions.展开更多
基金supported by Incheon NationalUniversity Research Grant in 2017.
文摘Video prediction is the problem of generating future frames by exploiting the spatiotemporal correlation from the past frame sequence.It is one of the crucial issues in computer vision and has many real-world applications,mainly focused on predicting future scenarios to avoid undesirable outcomes.However,modeling future image content and object is challenging due to the dynamic evolution and complexity of the scene,such as occlusions,camera movements,delay and illumination.Direct frame synthesis or optical-flow estimation are common approaches used by researchers.However,researchers mainly focused on video prediction using one of the approaches.Both methods have limitations,such as direct frame synthesis,usually face blurry prediction due to complex pixel distributions in the scene,and optical-flow estimation,usually produce artifacts due to large object displacements or obstructions in the clip.In this paper,we constructed a deep neural network Frame Prediction Network(FPNet-OF)with multiplebranch inputs(optical flow and original frame)to predict the future video frame by adaptively fusing the future object-motion with the future frame generator.The key idea is to jointly optimize direct RGB frame synthesis and dense optical flow estimation to generate a superior video prediction network.Using various real-world datasets,we experimentally verify that our proposed framework can produce high-level video frame compared to other state-ofthe-art framework.
文摘Video-based vibration measurement is a cost-effective way for remote monitoring the health conditions of transportation and other civil structures, especially for situations where accessibility is restricted and does not allow installation of conventional monitoring devices. Besides, video-based system is global measurement. The technical basis of video-based remote vibration measurement system is digital image analysis. Comparison of the images allow the field of motion to be accurately delineated. Such information is important to understand the structure behaviors including the motion and strain distribution. This paper presents system and analyses to monitor the vibration velocity and displacement field. The performance is demonstrated on a testbed of model building. Three different methods （i.e., frame difference method, particle image velocimetry, and optical Flow Method） are utilized to analyze the image sequences to extract the feature of motion. The Performance is validated using accelerometer data. The results indicate that all three methods can estimate the velocity field of the model building, although the results can be affected by factors such as background noise and environmental interference. Optical flow method achieved the best performance among these three methods studied. With further refinement of system hardware and image processing software, it will be developed into a remote video based monitoring system for structural health monitoring of transportation infrastructure to assist the diagnoses of its health conditions.
