Learning-based methods have proven successful in compressing geometric information for point clouds.For attribute compression,however,they still lag behind non-learning-based methods such as the MPEG G-PCC standard.To...Learning-based methods have proven successful in compressing geometric information for point clouds.For attribute compression,however,they still lag behind non-learning-based methods such as the MPEG G-PCC standard.To bridge this gap,we propose a novel deep learning-based point cloud attribute compression method that uses a generative adversarial network(GAN)with sparse convolution layers.Our method also includes a module that adaptively selects the resolution of the voxels used to voxelize the input point cloud.Sparse vectors are used to represent the voxelized point cloud,and sparse convolutions process the sparse tensors,ensuring computational efficiency.To the best of our knowledge,this is the first application of GANs to compress point cloud attributes.Our experimental results show that our method outperforms existing learning-based techniques and rivals the latest G-PCC test model(TMC13v23)in terms of visual quality.展开更多
Generative adversarial networks(GANs)are an unsupervised generative model that learns data distribution through adversarial training.However,recent experiments indicated that GANs are difficult to train due to the req...Generative adversarial networks(GANs)are an unsupervised generative model that learns data distribution through adversarial training.However,recent experiments indicated that GANs are difficult to train due to the requirement of optimization in the high dimensional parameter space and the zero gradient problem.In this work,we propose a self-sparse generative adversarial network(Self-Sparse GAN)that reduces the parameter space and alleviates the zero gradient problem.In the Self-Sparse GAN,we design a self-adaptive sparse transform module(SASTM)comprising the sparsity decomposition and feature-map recombination,which can be applied on multi-channel feature maps to obtain sparse feature maps.The key idea of Self-Sparse GAN is to add the SASTM following every deconvolution layer in the generator,which can adaptively reduce the parameter space by utilizing the sparsity in multi-channel feature maps.We theoretically prove that the SASTM can not only reduce the search space of the convolution kernel weight of the generator but also alleviate the zero gradient problem by maintaining meaningful features in the batch normalization layer and driving the weight of deconvolution layers away from being negative.The experimental results show that our method achieves the best Fréchet inception distance(FID)scores for image generation compared with Wasserstein GAN with gradient penalty(WGAN-GP)on MNIST,Fashion-MNIST,CIFAR-10,STL-10,mini-ImageNet,CELEBA-HQ,and LSUN bedrooms datasets,and the relative decrease of FID is 4.76%-21.84%.Meanwhile,an architectural sketch dataset(Sketch)is also used to validate the superiority of the proposed method.展开更多
相较于传统的无线电数据特征提取方法,深度学习具有高效灵活的特点,其可以有效提高调制数据识别的性能。然而在实践中,收集大量可靠的无线电调制样本数据有时代价是昂贵和困难的,这在很大程度上限制了深度学习模型的性能。本文提出了基...相较于传统的无线电数据特征提取方法,深度学习具有高效灵活的特点,其可以有效提高调制数据识别的性能。然而在实践中,收集大量可靠的无线电调制样本数据有时代价是昂贵和困难的,这在很大程度上限制了深度学习模型的性能。本文提出了基于生成对抗网络(Generative Adversarial Networks,GAN)的无线电调制数据增扩模型RMAbGAN(Radio Modulation data Augmentation based on Generative Adversarial Networks),该模型通过挖掘不同信噪比与调制方式下的无线电调制数据特征差异,生成符合调制方式与信噪比特点的无线电调制数据,模型中的生成器部分捕获无线电调制数据分布特征,辨识器部分优化生成器性能,两者相互博弈性能不断提升;在此基础上,对无线电数据采样特点与无线电数据传统增强方法进行深度分析与研究,发现了无线电调制数据蕴含的空域特征与时序特征,设计出了能深刻捕获无线电数据空域特征与时序特征的无线电数据分类模型AMCST(Automatic Modulation Classification based Spatial and Temporal feature)。通过大量的对比实验,表明相较于基于旋转变换的无线电调制数据增扩模型,RMAbGAN模型在无线电调制数据增扩方面更具有鲁棒性和泛化能力,可以实现更高的调制分类准确率。此外,相较基于长短期记忆网络(Long Short-Term Memory,LSTM)的调制分类模型、基于残差网络(Residual Networks,ResNet)的调制分类模型等传统模型,AMCST模型在调制数据分类方面更具有稳定性和可用性,同时也具有更高的分类准确率。展开更多
基金supported in part by the National Natural Science Foundation of China under Grants 62222110,62172259,62571303the High-end Foreign Experts Recruitment Plan of Chinese Ministry of Science and Technology under Grant G2023150003L+2 种基金the exchange project of the Royal Society and the National Natural Science Foundation of China under Grant 62311530104(IEC\NSFC\223076)the Taishan Scholar Project of Shandong Province(tsqn202103001)the Natural Science Foundation of Shandong Province under Grant ZR2022ZD38.
文摘Learning-based methods have proven successful in compressing geometric information for point clouds.For attribute compression,however,they still lag behind non-learning-based methods such as the MPEG G-PCC standard.To bridge this gap,we propose a novel deep learning-based point cloud attribute compression method that uses a generative adversarial network(GAN)with sparse convolution layers.Our method also includes a module that adaptively selects the resolution of the voxels used to voxelize the input point cloud.Sparse vectors are used to represent the voxelized point cloud,and sparse convolutions process the sparse tensors,ensuring computational efficiency.To the best of our knowledge,this is the first application of GANs to compress point cloud attributes.Our experimental results show that our method outperforms existing learning-based techniques and rivals the latest G-PCC test model(TMC13v23)in terms of visual quality.
基金This work was supported by the National Natural Science Foundation of China(Nos.51921006 and 52008138)Heilongjiang Touyan Innovation Team Program(No.AUEA5640200320).
文摘Generative adversarial networks(GANs)are an unsupervised generative model that learns data distribution through adversarial training.However,recent experiments indicated that GANs are difficult to train due to the requirement of optimization in the high dimensional parameter space and the zero gradient problem.In this work,we propose a self-sparse generative adversarial network(Self-Sparse GAN)that reduces the parameter space and alleviates the zero gradient problem.In the Self-Sparse GAN,we design a self-adaptive sparse transform module(SASTM)comprising the sparsity decomposition and feature-map recombination,which can be applied on multi-channel feature maps to obtain sparse feature maps.The key idea of Self-Sparse GAN is to add the SASTM following every deconvolution layer in the generator,which can adaptively reduce the parameter space by utilizing the sparsity in multi-channel feature maps.We theoretically prove that the SASTM can not only reduce the search space of the convolution kernel weight of the generator but also alleviate the zero gradient problem by maintaining meaningful features in the batch normalization layer and driving the weight of deconvolution layers away from being negative.The experimental results show that our method achieves the best Fréchet inception distance(FID)scores for image generation compared with Wasserstein GAN with gradient penalty(WGAN-GP)on MNIST,Fashion-MNIST,CIFAR-10,STL-10,mini-ImageNet,CELEBA-HQ,and LSUN bedrooms datasets,and the relative decrease of FID is 4.76%-21.84%.Meanwhile,an architectural sketch dataset(Sketch)is also used to validate the superiority of the proposed method.
文摘相较于传统的无线电数据特征提取方法,深度学习具有高效灵活的特点,其可以有效提高调制数据识别的性能。然而在实践中,收集大量可靠的无线电调制样本数据有时代价是昂贵和困难的,这在很大程度上限制了深度学习模型的性能。本文提出了基于生成对抗网络(Generative Adversarial Networks,GAN)的无线电调制数据增扩模型RMAbGAN(Radio Modulation data Augmentation based on Generative Adversarial Networks),该模型通过挖掘不同信噪比与调制方式下的无线电调制数据特征差异,生成符合调制方式与信噪比特点的无线电调制数据,模型中的生成器部分捕获无线电调制数据分布特征,辨识器部分优化生成器性能,两者相互博弈性能不断提升;在此基础上,对无线电数据采样特点与无线电数据传统增强方法进行深度分析与研究,发现了无线电调制数据蕴含的空域特征与时序特征,设计出了能深刻捕获无线电数据空域特征与时序特征的无线电数据分类模型AMCST(Automatic Modulation Classification based Spatial and Temporal feature)。通过大量的对比实验,表明相较于基于旋转变换的无线电调制数据增扩模型,RMAbGAN模型在无线电调制数据增扩方面更具有鲁棒性和泛化能力,可以实现更高的调制分类准确率。此外,相较基于长短期记忆网络(Long Short-Term Memory,LSTM)的调制分类模型、基于残差网络(Residual Networks,ResNet)的调制分类模型等传统模型,AMCST模型在调制数据分类方面更具有稳定性和可用性,同时也具有更高的分类准确率。
