The evolution of social network and multimedia technologies encourage more and more people to generate and upload visual information, which leads to the generation of large-scale video data. Therefore, preeminent comp...The evolution of social network and multimedia technologies encourage more and more people to generate and upload visual information, which leads to the generation of large-scale video data. Therefore, preeminent compression technologies are highly desired to facilitate the storage and transmission of these tremendous video data for a wide variety of applications. In this paper, a systematic review of the recent advances for large-scale video compression （LSVC） is presented. Specifically, fast video coding algorithms and effective models to improve video compression efficiency are introduced in detail, since coding complexity and compression efficiency are two important factors to evaluate video coding approaches. Finally, the challenges and fu- ture research trends for LSVC are discussed.展开更多
It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modu...It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modulate a recorded high-speed scene within one integration time.The superimposed image captured in this manner is modulated and compressed,since multiple modulation patterns are imposed.Following this,reconstruction algorithms are utilized to recover the desired high-speed scene.One leading advantage of video CS is that a single captured measurement can be used to reconstruct a multi-frame video,thereby enabling a low-speed camera to capture high-speed scenes.Inspired by this,a number of variants of video CS systems have been built,mainly using different modulation devices.Meanwhile,in order to obtain high-quality reconstruction videos,many algorithms have been developed,from optimization-based iterative algorithms to deep-learning-based ones.Recently,emerging deep learning methods have been dominant due to their high-speed inference and high-quality reconstruction,highlighting the possibility of deploying video CS in practical applications.Toward this end,this paper reviews the progress that has been achieved in video CS during the past decade.We further analyze the efforts that need to be made—in terms of both hardware and algorithms—to enable real applications.Research gaps are put forward and future directions are summarized to help researchers and engineers working on this topic.展开更多
High-resolution video transmission requires a substantial amount of bandwidth.In this paper,we present a novel video processing methodology that innovatively integrates region of interest(ROI)identification and super-...High-resolution video transmission requires a substantial amount of bandwidth.In this paper,we present a novel video processing methodology that innovatively integrates region of interest(ROI)identification and super-resolution enhancement.Our method commences with the accurate detection of ROIs within video sequences,followed by the application of advanced super-resolution techniques to these areas,thereby preserving visual quality while economizing on data transmission.To validate and benchmark our approach,we have curated a new gaming dataset tailored to evaluate the effectiveness of ROI-based super-resolution in practical applications.The proposed model architecture leverages the transformer network framework,guided by a carefully designed multi-task loss function,which facilitates concurrent learning and execution of both ROI identification and resolution enhancement tasks.This unified deep learning model exhibits remarkable performance in achieving super-resolution on our custom dataset.The implications of this research extend to optimizing low-bitrate video streaming scenarios.By selectively enhancing the resolution of critical regions in videos,our solution enables high-quality video delivery under constrained bandwidth conditions.Empirical results demonstrate a 15%reduction in transmission bandwidth compared to traditional super-resolution based compression methods,without any perceivable decline in visual quality.This work thus contributes to the advancement of video compression and enhancement technologies,offering an effective strategy for improving digital media delivery efficiency and user experience,especially in bandwidth-limited environments.The innovative integration of ROI identification and super-resolution presents promising avenues for future research and development in adaptive and intelligent video communication systems.展开更多
We introduce a factorized Smith method(FSM)for solving large-scale highranked T-Stein equations within the banded-plus-low-rank structure framework.To effectively reduce both computational complexity and storage requi...We introduce a factorized Smith method(FSM)for solving large-scale highranked T-Stein equations within the banded-plus-low-rank structure framework.To effectively reduce both computational complexity and storage requirements,we develop techniques including deflation and shift,partial truncation and compression,as well as redesign the residual computation and termination condition.Numerical examples demonstrate that the FSM outperforms the Smith method implemented with a hierarchical HODLR structured toolkit in terms of CPU time.展开更多
BACKGROUND It remains unclear whether video aids can improve the quality of bystander cardiopulmonary resuscitation(CPR).AIM To summarize simulation-based studies aiming at improving bystander CPR associated with the ...BACKGROUND It remains unclear whether video aids can improve the quality of bystander cardiopulmonary resuscitation(CPR).AIM To summarize simulation-based studies aiming at improving bystander CPR associated with the quality of chest compression and time-related quality parameters.METHODS The systematic review was conducted according to the PRISMA guidelines.All relevant studies were searched through PubMed,EMBASE,Medline and Cochrane Library databases.The risk of bias was evaluated using the Cochrane collaboration tool.RESULTS A total of 259 studies were eligible for inclusion,and 6 randomised controlled trial studies were ultimately included.The results of meta-analysis indicated that video-assisted CPR(V-CPR)was significantly associated with the improved mean chest compression rate[OR=0.66(0.49-0.82),P<0.001],and the proportion of chest compression with correct hand positioning[OR=1.63(0.71-2.55),P<0.001].However,the difference in mean chest compression depth was not statistically significant[OR=0.18(-0.07-0.42),P=0.15],and V-CPR was not associated with the time to first chest compression compared to telecommunicator CPR[OR=-0.12(-0.88-0.63),P=0.75].CONCLUSION Video real-time guidance by the dispatcher can improve the quality of bystander CPR to a certain extent.However,the quality is still not ideal,and there is a lack of guidance caused by poor video signal or inadequate interaction.展开更多
Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithm...Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithms restore image sequences of dynamic scenes, offering advantages such as reduced bandwidth and storage space requirements. The temporal correlation in video data is crucial for Video SCI, as it leverages the temporal relationships among frames to enhance the efficiency and quality of reconstruction algorithms, particularly for fast-moving objects.This paper discretizes video frames to create image datasets with the same data volume but differing temporal correlations. We utilized the state-of-the-art(SOTA) reconstruction framework, EfficientSCI++, to train various compressed reconstruction models with these differing temporal correlations. Evaluating the reconstruction results from these models, our simulation experiments confirm that a reduction in temporal correlation leads to decreased reconstruction accuracy. Additionally, we simulated the reconstruction outcomes of datasets devoid of temporal correlation, illustrating that models trained on non-temporal data affect the temporal feature extraction capabilities of transformers, resulting in negligible impacts on the evaluation of reconstruction results for non-temporal correlation test datasets.展开更多
A layered compression algorithm is presented which delivers spatial scalable encoded bit streams for remote video monitoring system. The complexity of the algorithm is modest and is well suited to real time implementa...A layered compression algorithm is presented which delivers spatial scalable encoded bit streams for remote video monitoring system. The complexity of the algorithm is modest and is well suited to real time implementation. Based on the layered compression algorithm, a codec system model is established. High-speed video compression can be realized with parallel data compression in this codec system. For image reconstruction, a prediction method using two nearest pix points is presented.展开更多
In this paper, we present a method using video codec technology to compress ECG signals. This method exploits both intra-beat and inter-beat correlations of the ECG signals to achieve high compression ratios (CR) and ...In this paper, we present a method using video codec technology to compress ECG signals. This method exploits both intra-beat and inter-beat correlations of the ECG signals to achieve high compression ratios (CR) and a low percent root mean square difference (PRD). Since ECG signals have both intra-beat and inter-beat redundancies like video signals, which have both intra-frame and inter-frame correlation, video codec technology can be used for ECG compression. In order to do this, some pre-process will be needed. The ECG signals should firstly be segmented and normalized to a sequence of beat cycles with the same length, and then these beat cycles can be treated as picture frames and compressed with video codec technology. We have used records from MIT-BIH arrhythmia database to evaluate our algorithm. Results show that, besides compression efficiently, this algorithm has the advantages of resolution adjustable, random access and flexibility for irregular period and QRS false detection.展开更多
In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D vi...In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.展开更多
In this paper, a new mesh based algorithm is applied for motion estimation and compensation in the wavelet domain. The first major contribution of this work is the introduction of a new active mesh based method for mo...In this paper, a new mesh based algorithm is applied for motion estimation and compensation in the wavelet domain. The first major contribution of this work is the introduction of a new active mesh based method for motion estimation and compensation. The proposed algorithm is based on the mesh energy minimization with novel sets of energy functions. The proposed energy functions have appropriate features, which improve the accuracy of motion estimation and compensation algorithm. We employ the proposed motion estimation algorithm in two different manners for video compression. In the first approach, the proposed algorithm is employed for motion estimation of consecutive frames. In the second approach, the algorithm is applied for motion estimation and compensation in the wavelet sub-bands. The experimental results reveal that the incorporation of active mesh based motion-compensated temporal filtering into wavelet sub-bands significantly improves the distortion performance rate of the video compression. We also use a new wavelet coder for the coding of the 3D volume of coefficients based on the retained energy criteria. This coder gives the maximum retained energy in all sub-bands. The proposed algorithm was tested with some video sequences and the results showed that the use of the proposed active mesh method for motion compensation and its implementation in sub-bands yields significant improvement in PSNR performance.展开更多
Multimedia semantic communication has been receiving increasing attention due to its significant enhancement of communication efficiency.Semantic coding,which is oriented towards extracting and encoding the key semant...Multimedia semantic communication has been receiving increasing attention due to its significant enhancement of communication efficiency.Semantic coding,which is oriented towards extracting and encoding the key semantics of video for transmission,is a key aspect in the framework of multimedia semantic communication.In this paper,we propose a facial video semantic coding method with low bitrate based on the temporal continuity of video semantics.At the sender’s end,we selectively transmit facial keypoints and deformation information,allocating distinct bitrates to different keypoints across frames.Compressive techniques involving sampling and quantization are employed to reduce the bitrate while retaining facial key semantic information.At the receiver’s end,a GAN-based generative network is utilized for reconstruction,effectively mitigating block artifacts and buffering problems present in traditional codec algorithms under low bitrates.The performance of the proposed approach is validated on multiple datasets,such as VoxCeleb and TalkingHead-1kH,employing metrics such as LPIPS,DISTS,and AKD for assessment.Experimental results demonstrate significant advantages over traditional codec methods,achieving up to approximately 10-fold bitrate reduction in prolonged,stable head pose scenarios across diverse conversational video settings.展开更多
High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We int...High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We introduce“SpeedShot,”a computational imaging framework for efficient high-speed video imaging.SpeedShot features a low-speed dual-camera setup,which simultaneously captures two temporally coded snapshots.Cross-referencing these two snapshots extracts a multiplexed temporal gradient image,producing a compact and multiframe motion representation for video reconstruction.Recognizing the unique temporal-only modulation model,we propose an explicable motion-guided scale-recurrent transformer for video decoding.It exploits cross-scale error maps to bolster the cycle consistency between predicted and observed data.Evaluations on both simulated datasets and real imaging setups demonstrate SpeedShot’s effectiveness in video-rate up-conversion,with pronounced improvement over video frame interpolation and deblurring methods.The proposed framework is compatible with commercial low-speed cameras,offering a versatile low-bandwidth alternative for video-related applications,such as video surveillance and sports analysis.展开更多
The particle image velocimetry (PIV) method was used to investigate the full-field displacements and strains of the limestone specimen under external loads from the video images captured during the laboratory tests.Th...The particle image velocimetry (PIV) method was used to investigate the full-field displacements and strains of the limestone specimen under external loads from the video images captured during the laboratory tests.The original colorful video images and experimental data were obtained from the uniaxial compression test of a limestone.To eliminate perspective errors and lens distortion,the camera was placed normal to the rock specimen exposure.After converted into a readable format of frame images,these videos were transformed into the responding grayscale images,and the frame images were then extracted.The full-field displacement field was obtained by using the PIV technique,and interpolated in the sub-pixel locations.The displacement was measured in the plane of the image and inferred from two consecutive images.The local displacement vectors were calculated for small sub-windows of the images by means of cross-correlation.The video images were interrogated in a multi-pass way,starting off with 64×64 images,ending with 16×16 images after 6 iterations,and using 75% overlap of the sub-windows.In order to remove spurious vectors,the displacements were filtered using four filters:signal-to-noise ratio filter,peak height filter,global filter and local filter.The cubic interpolation was utilized if the displacements without a number were encountered.The full-field strain was then obtained using the local least square method from the discrete displacements.The strain change with time at different locations was also investigated.It is found that the normal strains are dependant on the locations and the crack distributions.Between 1.0 and 5.0 s prior to the specimen failure,normal strains increase rapidly at many locations,while a stable status appears at some locations.When the specimen is in a failure status,a large rotation occurs and it increases in the inverse direction.The strain concentration bands do not completely develop into the large cracks,and meso-cracks are not visible in some bands.The techniques presented here may improve the traditional measurement of the strain field,and may provide a lot of valuable information in investigating the deformation/failure mechanism of rock materials.展开更多
An edge oriented image sequence coding scheme is presented. On the basis of edge detecting, an image could be divided into the sensitized region and the smooth region. In this scheme, the architecture of sensitized r...An edge oriented image sequence coding scheme is presented. On the basis of edge detecting, an image could be divided into the sensitized region and the smooth region. In this scheme, the architecture of sensitized region is approximated with linear type of segments. Then a rectangle belt is constructed for each segment. Finally, the gray value distribution in the region is fitted by normal forms polynomials. The model matching and motion analysis are also based on the architecture of sensitized region. For the smooth region we use the run length scanning and linear approximating. By means of normal forms polynomial fitting and motion prediction by matching, the images are compressed. It is shown through the simulations that the subjective quality of reconstructed picture is excellent at 0.0075 bit per pel.展开更多
The exponential growth of cultural heritage documentation videos calls for new compression methods that preserve critical details while reducing storage.For static scenes,traditional frame-based compression methods st...The exponential growth of cultural heritage documentation videos calls for new compression methods that preserve critical details while reducing storage.For static scenes,traditional frame-based compression methods struggle with the trade-off between semantic redundancy and detail preservation.To improve compression efficiency,a novel dual-mode semantic compression framework for static object videos based on neural radiance fields(NeRF)was proposed in this paper.By integrating semantic segmentation with COLMAP technology,the proposed system decouples the video stream into two semantic layers,which are the central object containing critical details and the dynamic background rich in semantic redundancy,respectively.In the proposed dual-mode framework,the focus-priority(FP)mode is designed for scenarios with high-efficiency demands,where only the NeRF-based neural representation of the primary object is preserved and compressed.For scenarios that require additional environmental context,the panorama-compatible(PC)mode synchronously compresses the H.264-encoded background streams and the primary object streams to reconstruct the full scene.Experimental results on single-artifact video data demonstrate that the proposed framework achieves a storage reduction of 20%compared with conventional methods,thus providing a flexible and controllable solution for the compression of cultural heritage documentation videos.展开更多
Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random ...Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random matrix destroys the target location information in the original video frames. This paper summarizes our research effort on target tracking and classification directly in the compressive measurement domain. We focus on one particular type of compressive measurement using pixel subsampling. That is, original pixels in video frames are randomly subsampled. Even in such a special compressive sensing setting, conventional trackers do not work in a satisfactory manner. We propose a deep learning approach that integrates YOLO (You Only Look Once) and ResNet (residual network) for multiple target tracking and classification. YOLO is used for multiple target tracking and ResNet is for target classification. Extensive experiments using short wave infrared (SWIR), mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demonstrated the efficacy of the proposed approach even though the training data are very scarce.展开更多
Super-Resolution (SR) technique means to reconstruct High-Resolution (HR) images from a sequence of Low-Resolution (LR) observations,which has been a great focus for compressed video. Based on the theory of Projection...Super-Resolution (SR) technique means to reconstruct High-Resolution (HR) images from a sequence of Low-Resolution (LR) observations,which has been a great focus for compressed video. Based on the theory of Projection Onto Convex Set (POCS),this paper constructs Quantization Constraint Set (QCS) using the quantization information extracted from the video bit stream. By combining the statistical properties of image and the Human Visual System (HVS),a novel Adaptive Quantization Constraint Set (AQCS) is proposed. Simulation results show that AQCS-based SR al-gorithm converges at a fast rate and obtains better performance in both objective and subjective quality,which is applicable for compressed video.展开更多
In this paper,a video compressed sensing reconstruction algorithm based on multidimensional reference frames is proposed using the sparse characteristics of video signals in different sparse representation domains.Fir...In this paper,a video compressed sensing reconstruction algorithm based on multidimensional reference frames is proposed using the sparse characteristics of video signals in different sparse representation domains.First,the overall structure of the proposed video compressed sensing algorithm is introduced in this paper.The paper adopts a multi-reference frame bidirectional prediction hypothesis optimization algorithm.Then,the paper proposes a reconstruction method for CS frames at the re-decoding end.In addition to using key frames of each GOP reconstructed in the time domain as reference frames for reconstructing CS frames,half-pixel reference frames and scaled reference frames in the pixel domain are also used as CS frames.Reference frames of CS frames are used to obtain higher quality assumptions.Themethod of obtaining reference frames in the pixel domain is also discussed in detail in this paper.Finally,the reconstruction algorithm proposed in this paper is compared with video compression algorithms in the literature that have better reconstruction results.Experiments show that the algorithm has better performance than the best multi-reference frame video compression sensing algorithm and can effectively improve the quality of slowmotion video reconstruction.展开更多
基金This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 61622115 and 61472281）, the Program for Professor of Special Appointment （Eastern Scholar） at Shanghai Institutions of Higher Learning （GZ2015005）, and Shanghai Engineering Research Center of Industrial Vision Perception ＆ Intelligent Computing （ 17DZ2251600）.
文摘The evolution of social network and multimedia technologies encourage more and more people to generate and upload visual information, which leads to the generation of large-scale video data. Therefore, preeminent compression technologies are highly desired to facilitate the storage and transmission of these tremendous video data for a wide variety of applications. In this paper, a systematic review of the recent advances for large-scale video compression （LSVC） is presented. Specifically, fast video coding algorithms and effective models to improve video compression efficiency are introduced in detail, since coding complexity and compression efficiency are two important factors to evaluate video coding approaches. Finally, the challenges and fu- ture research trends for LSVC are discussed.
基金supported by the National Natural Science Foundation of China(61931012,62171258,62088102,and 62271414)the Zhejiang Provincial Outstanding Youth Science Foundation(LR23F010001)the Key Project of Westlake Institute for Optoelectronics(2023GD007).
文摘It has been over a decade since the first coded aperture video compressive sensing(CS)system was reported.The underlying principle of this technology is to employ a high-frequency modulator in the optical path to modulate a recorded high-speed scene within one integration time.The superimposed image captured in this manner is modulated and compressed,since multiple modulation patterns are imposed.Following this,reconstruction algorithms are utilized to recover the desired high-speed scene.One leading advantage of video CS is that a single captured measurement can be used to reconstruct a multi-frame video,thereby enabling a low-speed camera to capture high-speed scenes.Inspired by this,a number of variants of video CS systems have been built,mainly using different modulation devices.Meanwhile,in order to obtain high-quality reconstruction videos,many algorithms have been developed,from optimization-based iterative algorithms to deep-learning-based ones.Recently,emerging deep learning methods have been dominant due to their high-speed inference and high-quality reconstruction,highlighting the possibility of deploying video CS in practical applications.Toward this end,this paper reviews the progress that has been achieved in video CS during the past decade.We further analyze the efforts that need to be made—in terms of both hardware and algorithms—to enable real applications.Research gaps are put forward and future directions are summarized to help researchers and engineers working on this topic.
基金funded by National Key Research and Development Program of China(No.2022YFC3302103).
文摘High-resolution video transmission requires a substantial amount of bandwidth.In this paper,we present a novel video processing methodology that innovatively integrates region of interest(ROI)identification and super-resolution enhancement.Our method commences with the accurate detection of ROIs within video sequences,followed by the application of advanced super-resolution techniques to these areas,thereby preserving visual quality while economizing on data transmission.To validate and benchmark our approach,we have curated a new gaming dataset tailored to evaluate the effectiveness of ROI-based super-resolution in practical applications.The proposed model architecture leverages the transformer network framework,guided by a carefully designed multi-task loss function,which facilitates concurrent learning and execution of both ROI identification and resolution enhancement tasks.This unified deep learning model exhibits remarkable performance in achieving super-resolution on our custom dataset.The implications of this research extend to optimizing low-bitrate video streaming scenarios.By selectively enhancing the resolution of critical regions in videos,our solution enables high-quality video delivery under constrained bandwidth conditions.Empirical results demonstrate a 15%reduction in transmission bandwidth compared to traditional super-resolution based compression methods,without any perceivable decline in visual quality.This work thus contributes to the advancement of video compression and enhancement technologies,offering an effective strategy for improving digital media delivery efficiency and user experience,especially in bandwidth-limited environments.The innovative integration of ROI identification and super-resolution presents promising avenues for future research and development in adaptive and intelligent video communication systems.
基金Supported partly by NSF of China(Grant No.11801163)NSF of Hunan Province(Grant Nos.2021JJ50032,2023JJ50164 and 2023JJ50165)Degree&Postgraduate Reform Project of Hunan University of Technology and Hunan Province(Grant Nos.JGYB23009 and 2024JGYB210).
文摘We introduce a factorized Smith method(FSM)for solving large-scale highranked T-Stein equations within the banded-plus-low-rank structure framework.To effectively reduce both computational complexity and storage requirements,we develop techniques including deflation and shift,partial truncation and compression,as well as redesign the residual computation and termination condition.Numerical examples demonstrate that the FSM outperforms the Smith method implemented with a hierarchical HODLR structured toolkit in terms of CPU time.
基金Supported by the Fundamental Research Funds for the Central Universities,Northwest Minzu University,Grant No.31920170180.
文摘BACKGROUND It remains unclear whether video aids can improve the quality of bystander cardiopulmonary resuscitation(CPR).AIM To summarize simulation-based studies aiming at improving bystander CPR associated with the quality of chest compression and time-related quality parameters.METHODS The systematic review was conducted according to the PRISMA guidelines.All relevant studies were searched through PubMed,EMBASE,Medline and Cochrane Library databases.The risk of bias was evaluated using the Cochrane collaboration tool.RESULTS A total of 259 studies were eligible for inclusion,and 6 randomised controlled trial studies were ultimately included.The results of meta-analysis indicated that video-assisted CPR(V-CPR)was significantly associated with the improved mean chest compression rate[OR=0.66(0.49-0.82),P<0.001],and the proportion of chest compression with correct hand positioning[OR=1.63(0.71-2.55),P<0.001].However,the difference in mean chest compression depth was not statistically significant[OR=0.18(-0.07-0.42),P=0.15],and V-CPR was not associated with the time to first chest compression compared to telecommunicator CPR[OR=-0.12(-0.88-0.63),P=0.75].CONCLUSION Video real-time guidance by the dispatcher can improve the quality of bystander CPR to a certain extent.However,the quality is still not ideal,and there is a lack of guidance caused by poor video signal or inadequate interaction.
基金supported in part by the National Natural Science Foundation of China (No. U23B2011)。
文摘Video snapshot compressive imaging(Video SCI) modulates scenes using various encoding masks and captures compressed measurements with a low-speed camera during a single exposure. Subsequently, reconstruction algorithms restore image sequences of dynamic scenes, offering advantages such as reduced bandwidth and storage space requirements. The temporal correlation in video data is crucial for Video SCI, as it leverages the temporal relationships among frames to enhance the efficiency and quality of reconstruction algorithms, particularly for fast-moving objects.This paper discretizes video frames to create image datasets with the same data volume but differing temporal correlations. We utilized the state-of-the-art(SOTA) reconstruction framework, EfficientSCI++, to train various compressed reconstruction models with these differing temporal correlations. Evaluating the reconstruction results from these models, our simulation experiments confirm that a reduction in temporal correlation leads to decreased reconstruction accuracy. Additionally, we simulated the reconstruction outcomes of datasets devoid of temporal correlation, illustrating that models trained on non-temporal data affect the temporal feature extraction capabilities of transformers, resulting in negligible impacts on the evaluation of reconstruction results for non-temporal correlation test datasets.
文摘A layered compression algorithm is presented which delivers spatial scalable encoded bit streams for remote video monitoring system. The complexity of the algorithm is modest and is well suited to real time implementation. Based on the layered compression algorithm, a codec system model is established. High-speed video compression can be realized with parallel data compression in this codec system. For image reconstruction, a prediction method using two nearest pix points is presented.
文摘In this paper, we present a method using video codec technology to compress ECG signals. This method exploits both intra-beat and inter-beat correlations of the ECG signals to achieve high compression ratios (CR) and a low percent root mean square difference (PRD). Since ECG signals have both intra-beat and inter-beat redundancies like video signals, which have both intra-frame and inter-frame correlation, video codec technology can be used for ECG compression. In order to do this, some pre-process will be needed. The ECG signals should firstly be segmented and normalized to a sequence of beat cycles with the same length, and then these beat cycles can be treated as picture frames and compressed with video codec technology. We have used records from MIT-BIH arrhythmia database to evaluate our algorithm. Results show that, besides compression efficiently, this algorithm has the advantages of resolution adjustable, random access and flexibility for irregular period and QRS false detection.
文摘In this paper, we summarize 3D perception-oriented algorithms for perceptually driven 3D video coding. Several perceptual ef- fects have been exploited for 2D video viewing; however, this is not yet the case for 3D video viewing. 3D video requires depth perception, which implies binocular effects such as con fl icts, fusion, and rivalry. A better understanding of these effects is necessary for 3D perceptual compression, which provides users with a more comfortable visual experience for video that is de- livered over a channel with limited bandwidth. We present state-of-the-art of 3D visual attention models, 3D just-notice- able difference models, and 3D texture-synthesis models that address 3D human vision issues in 3D video coding and trans-mission.
文摘In this paper, a new mesh based algorithm is applied for motion estimation and compensation in the wavelet domain. The first major contribution of this work is the introduction of a new active mesh based method for motion estimation and compensation. The proposed algorithm is based on the mesh energy minimization with novel sets of energy functions. The proposed energy functions have appropriate features, which improve the accuracy of motion estimation and compensation algorithm. We employ the proposed motion estimation algorithm in two different manners for video compression. In the first approach, the proposed algorithm is employed for motion estimation of consecutive frames. In the second approach, the algorithm is applied for motion estimation and compensation in the wavelet sub-bands. The experimental results reveal that the incorporation of active mesh based motion-compensated temporal filtering into wavelet sub-bands significantly improves the distortion performance rate of the video compression. We also use a new wavelet coder for the coding of the 3D volume of coefficients based on the retained energy criteria. This coder gives the maximum retained energy in all sub-bands. The proposed algorithm was tested with some video sequences and the results showed that the use of the proposed active mesh method for motion compensation and its implementation in sub-bands yields significant improvement in PSNR performance.
基金supported by the National Natural Science Foundation of China (Nos. NSFC 61925105, 62322109, 62171257 and U22B2001)the Xplorer Prize in Information and Electronics technologiesthe Tsinghua University (Department of Electronic Engineering)-Nantong Research Institute for Advanced Communication Technologies Joint Research Center for Space, Air, Ground and Sea Cooperative Communication Network Technology
文摘Multimedia semantic communication has been receiving increasing attention due to its significant enhancement of communication efficiency.Semantic coding,which is oriented towards extracting and encoding the key semantics of video for transmission,is a key aspect in the framework of multimedia semantic communication.In this paper,we propose a facial video semantic coding method with low bitrate based on the temporal continuity of video semantics.At the sender’s end,we selectively transmit facial keypoints and deformation information,allocating distinct bitrates to different keypoints across frames.Compressive techniques involving sampling and quantization are employed to reduce the bitrate while retaining facial key semantic information.At the receiver’s end,a GAN-based generative network is utilized for reconstruction,effectively mitigating block artifacts and buffering problems present in traditional codec algorithms under low bitrates.The performance of the proposed approach is validated on multiple datasets,such as VoxCeleb and TalkingHead-1kH,employing metrics such as LPIPS,DISTS,and AKD for assessment.Experimental results demonstrate significant advantages over traditional codec methods,achieving up to approximately 10-fold bitrate reduction in prolonged,stable head pose scenarios across diverse conversational video settings.
基金supported by the National Natural Science Foundation of China(Grant No.62305184)the Basic and Applied Basic Research Foundation of Guangdong Province(Grant No.2023A1515012932)+7 种基金the Science,Technology,and Innovation Commission of Shenzhen Municipality(Grant No.JCYJ20241202123919027)the Major Key Project of Pengcheng Laboratory(Grant No.PCL2024A1)the Science Fund for Distinguished Young Scholars of Zhejiang Province(Grant No.LR23F010001)the Research Center for Industries of the Future(RCIF)at Westlake University and and the Key Project of Westlake Institute for Optoelectronics(Grant No.2023GD007)the Zhejiang“Pioneer”and“Leading Goose”R&D Program(Grant Nos.2024SDXHDX0006 and 2024C03182)the Ningbo Science and Technology Bureau“Science and Technology Yongjiang 2035”Key Technology Breakthrough Program(Grant No.2024Z126)the Research Grants Council of the Hong Kong Special Administrative Region,China(Grant Nos.C5031-22G,CityU11310522,and CityU11300123)the City University of Hong Kong(Grant No.9610628).
文摘High-speed imaging is crucial for understanding the transient dynamics of the world,but conventional frame-by-frame video acquisition is limited by specialized hardware and substantial data storage requirements.We introduce“SpeedShot,”a computational imaging framework for efficient high-speed video imaging.SpeedShot features a low-speed dual-camera setup,which simultaneously captures two temporally coded snapshots.Cross-referencing these two snapshots extracts a multiplexed temporal gradient image,producing a compact and multiframe motion representation for video reconstruction.Recognizing the unique temporal-only modulation model,we propose an explicable motion-guided scale-recurrent transformer for video decoding.It exploits cross-scale error maps to bolster the cycle consistency between predicted and observed data.Evaluations on both simulated datasets and real imaging setups demonstrate SpeedShot’s effectiveness in video-rate up-conversion,with pronounced improvement over video frame interpolation and deblurring methods.The proposed framework is compatible with commercial low-speed cameras,offering a versatile low-bandwidth alternative for video-related applications,such as video surveillance and sports analysis.
基金Project(40972191) supported by the National Natural Science Foundation of ChinaProject(09YZ39) supported by the Creative Issue of Shanghai Education Committee,China
文摘The particle image velocimetry (PIV) method was used to investigate the full-field displacements and strains of the limestone specimen under external loads from the video images captured during the laboratory tests.The original colorful video images and experimental data were obtained from the uniaxial compression test of a limestone.To eliminate perspective errors and lens distortion,the camera was placed normal to the rock specimen exposure.After converted into a readable format of frame images,these videos were transformed into the responding grayscale images,and the frame images were then extracted.The full-field displacement field was obtained by using the PIV technique,and interpolated in the sub-pixel locations.The displacement was measured in the plane of the image and inferred from two consecutive images.The local displacement vectors were calculated for small sub-windows of the images by means of cross-correlation.The video images were interrogated in a multi-pass way,starting off with 64×64 images,ending with 16×16 images after 6 iterations,and using 75% overlap of the sub-windows.In order to remove spurious vectors,the displacements were filtered using four filters:signal-to-noise ratio filter,peak height filter,global filter and local filter.The cubic interpolation was utilized if the displacements without a number were encountered.The full-field strain was then obtained using the local least square method from the discrete displacements.The strain change with time at different locations was also investigated.It is found that the normal strains are dependant on the locations and the crack distributions.Between 1.0 and 5.0 s prior to the specimen failure,normal strains increase rapidly at many locations,while a stable status appears at some locations.When the specimen is in a failure status,a large rotation occurs and it increases in the inverse direction.The strain concentration bands do not completely develop into the large cracks,and meso-cracks are not visible in some bands.The techniques presented here may improve the traditional measurement of the strain field,and may provide a lot of valuable information in investigating the deformation/failure mechanism of rock materials.
文摘An edge oriented image sequence coding scheme is presented. On the basis of edge detecting, an image could be divided into the sensitized region and the smooth region. In this scheme, the architecture of sensitized region is approximated with linear type of segments. Then a rectangle belt is constructed for each segment. Finally, the gray value distribution in the region is fitted by normal forms polynomials. The model matching and motion analysis are also based on the architecture of sensitized region. For the smooth region we use the run length scanning and linear approximating. By means of normal forms polynomial fitting and motion prediction by matching, the images are compressed. It is shown through the simulations that the subjective quality of reconstructed picture is excellent at 0.0075 bit per pel.
基金supported by the National Key Research and Development Program of China(2022YFB2902100)。
文摘The exponential growth of cultural heritage documentation videos calls for new compression methods that preserve critical details while reducing storage.For static scenes,traditional frame-based compression methods struggle with the trade-off between semantic redundancy and detail preservation.To improve compression efficiency,a novel dual-mode semantic compression framework for static object videos based on neural radiance fields(NeRF)was proposed in this paper.By integrating semantic segmentation with COLMAP technology,the proposed system decouples the video stream into two semantic layers,which are the central object containing critical details and the dynamic background rich in semantic redundancy,respectively.In the proposed dual-mode framework,the focus-priority(FP)mode is designed for scenarios with high-efficiency demands,where only the NeRF-based neural representation of the primary object is preserved and compressed.For scenarios that require additional environmental context,the panorama-compatible(PC)mode synchronously compresses the H.264-encoded background streams and the primary object streams to reconstruct the full scene.Experimental results on single-artifact video data demonstrate that the proposed framework achieves a storage reduction of 20%compared with conventional methods,thus providing a flexible and controllable solution for the compression of cultural heritage documentation videos.
文摘Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random matrix destroys the target location information in the original video frames. This paper summarizes our research effort on target tracking and classification directly in the compressive measurement domain. We focus on one particular type of compressive measurement using pixel subsampling. That is, original pixels in video frames are randomly subsampled. Even in such a special compressive sensing setting, conventional trackers do not work in a satisfactory manner. We propose a deep learning approach that integrates YOLO (You Only Look Once) and ResNet (residual network) for multiple target tracking and classification. YOLO is used for multiple target tracking and ResNet is for target classification. Extensive experiments using short wave infrared (SWIR), mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demonstrated the efficacy of the proposed approach even though the training data are very scarce.
基金Supported by National Natural Science Foundation of China（61170147） Major Cooperation Project of Production and College in Fujian Province（2012H61010016） Natural Science Foundation of Fujian Province（2013J01234）
基金the Natural Science Foundation of Jiangsu Province (No.BK2004151).
文摘Super-Resolution (SR) technique means to reconstruct High-Resolution (HR) images from a sequence of Low-Resolution (LR) observations,which has been a great focus for compressed video. Based on the theory of Projection Onto Convex Set (POCS),this paper constructs Quantization Constraint Set (QCS) using the quantization information extracted from the video bit stream. By combining the statistical properties of image and the Human Visual System (HVS),a novel Adaptive Quantization Constraint Set (AQCS) is proposed. Simulation results show that AQCS-based SR al-gorithm converges at a fast rate and obtains better performance in both objective and subjective quality,which is applicable for compressed video.
文摘In this paper,a video compressed sensing reconstruction algorithm based on multidimensional reference frames is proposed using the sparse characteristics of video signals in different sparse representation domains.First,the overall structure of the proposed video compressed sensing algorithm is introduced in this paper.The paper adopts a multi-reference frame bidirectional prediction hypothesis optimization algorithm.Then,the paper proposes a reconstruction method for CS frames at the re-decoding end.In addition to using key frames of each GOP reconstructed in the time domain as reference frames for reconstructing CS frames,half-pixel reference frames and scaled reference frames in the pixel domain are also used as CS frames.Reference frames of CS frames are used to obtain higher quality assumptions.Themethod of obtaining reference frames in the pixel domain is also discussed in detail in this paper.Finally,the reconstruction algorithm proposed in this paper is compared with video compression algorithms in the literature that have better reconstruction results.Experiments show that the algorithm has better performance than the best multi-reference frame video compression sensing algorithm and can effectively improve the quality of slowmotion video reconstruction.
