Fusion methods based on multi-scale transforms have become the mainstream of the pixel-level image fusion. However,most of these methods cannot fully exploit spatial domain information of source images, which lead to ...Fusion methods based on multi-scale transforms have become the mainstream of the pixel-level image fusion. However,most of these methods cannot fully exploit spatial domain information of source images, which lead to the degradation of image.This paper presents a fusion framework based on block-matching and 3D(BM3D) multi-scale transform. The algorithm first divides the image into different blocks and groups these 2D image blocks into 3D arrays by their similarity. Then it uses a 3D transform which consists of a 2D multi-scale and a 1D transform to transfer the arrays into transform coefficients, and then the obtained low-and high-coefficients are fused by different fusion rules. The final fused image is obtained from a series of fused 3D image block groups after the inverse transform by using an aggregation process. In the experimental part, we comparatively analyze some existing algorithms and the using of different transforms, e.g. non-subsampled Contourlet transform(NSCT), non-subsampled Shearlet transform(NSST), in the 3D transform step. Experimental results show that the proposed fusion framework can not only improve subjective visual effect, but also obtain better objective evaluation criteria than state-of-the-art methods.展开更多
为解决基于Transformer模型的配准方法不能有效表示待配准图像的全局和局部特征、解码器无法有效还原全局上下文信息等问题,基于Swin-Transformer和UNet模型,对Swin-UNet网络架构进行扩展,增强其表示全局和局部特征的能力.并且将最新的...为解决基于Transformer模型的配准方法不能有效表示待配准图像的全局和局部特征、解码器无法有效还原全局上下文信息等问题,基于Swin-Transformer和UNet模型,对Swin-UNet网络架构进行扩展,增强其表示全局和局部特征的能力.并且将最新的动态蛇形卷积推广到三维空间,建立3D DSC-STUNet(Swin TransformerUNet with 3D Dynamic Snake Convolution)模型用于图像配准.相比传统卷积方法,三维动态蛇形卷积能利用可变卷积核能更好地关注图像的局部细节,提高网络处理局部特征的能力.所提出的配准方法利用特征变换提取图像特征来生成形变场,并在此基础上以无监督的方式生成配准图像.在IXI(脑部信息数据集)和OASIS(开源影像数据集)脑部MRI(核磁共振成像)图像数据集上与五种经典的配准方法相比,实验结果表明:提出的方法能够得到更好的配准结果,Dice(戴斯相似系数)值最高可以提高18.1%.展开更多
当前,步态识别的主流方法常依赖堆叠卷积层来逐步扩大感受野,以融合局部特征,这种方法大多采用浅层网络,在提取步态图像的全局特征时存在一定的局限性,并缺乏对时序周期特征信息的关注。因此提出一种融合Transformer和3D卷积的深层神经...当前,步态识别的主流方法常依赖堆叠卷积层来逐步扩大感受野,以融合局部特征,这种方法大多采用浅层网络,在提取步态图像的全局特征时存在一定的局限性,并缺乏对时序周期特征信息的关注。因此提出一种融合Transformer和3D卷积的深层神经网络算法(3D convolutional gait recognition network based on adaptFormer and spect-conv,3D-ASgaitNet)。首先,初始残差卷积层将二进制轮廓数据转换为浮点编码特征图,以提供密集的低级结构特征;在此基础上,光谱层通过频域和时域的联合处理增强特征提取能力,并使用伪3D残差卷积模块进一步提取高级时空特征;最后融合AdaptFormer模块,通过轻量级的下采样-上采样网络结构,以适应不同的数据分布和任务需求,提供灵活的特征变换能力。3D-ASgaitNet分别在4个公开的室内数据集(CASIA-B、OU-MVLP)、室外数据集(GREW、Gait3D)上进行,分别取得99.84%、87.83%、45.32%、72.12%的识别准确率。实验结果表明,所提出方法在CASIA-B、Gait3D数据集中的识别准确率接近SOTA性能。展开更多
【背景】传统方法因静态感受野设计较难适配城市自动驾驶场景中汽车、行人及骑行者等目标的显著尺度差异,且跨尺度特征融合易引发层级干扰。【方法】针对自动驾驶场景中多类别、多尺寸目标的3D检测中跨尺度表征一致性的关键挑战,本研究...【背景】传统方法因静态感受野设计较难适配城市自动驾驶场景中汽车、行人及骑行者等目标的显著尺度差异,且跨尺度特征融合易引发层级干扰。【方法】针对自动驾驶场景中多类别、多尺寸目标的3D检测中跨尺度表征一致性的关键挑战,本研究提出基于均衡化感受野的3D目标检测方法VoxTNT,通过局部-全局协同注意力机制提升检测性能。在局部层面,设计了PointSetFormer模块,引入诱导集注意力模块(Induced Set Attention Block,ISAB),通过约简的交叉注意力聚合高密度点云的细粒度几何特征,突破传统体素均值池化的信息损失瓶颈;在全局层面,设计了VoxelFormerFFN模块,将非空体素抽象为超点集并实施跨体素ISAB交互,建立长程上下文依赖关系,并将全局特征学习计算负载从O(N^(2))压缩至O(M^(2))(M<<N,M为非空体素数量),规避了复杂的Transformer直接使用在原始点云造成的高计算复杂度。该双域耦合架构实现了局部细粒度感知与全局语义关联的动态平衡,有效缓解固定感受野和多尺度融合导致的特征建模偏差。【结果】实验表明,该方法在KITTI数据集单阶段检测下,中等难度级别的行人检测精度AP(Average Precision)值达到59.56%,较SECOND基线提高约12.4%,两阶段检测下以66.54%的综合指标mAP(mean Average Precision)领先次优方法BSAODet的66.10%。同时,在WOD数据集中验证了方法的有效性,综合指标mAP达到66.09%分别超越SECOND和PointPillars基线7.7%和8.5%。消融实验进一步表明,均衡化局部和全局感受野的3D特征学习机制能显著提升小目标检测精度(如在KITTI数据集中全组件消融的情况下,中等难度级别的行人和骑行者检测精度分别下降10.8%和10.0%),同时保持大目标检测的稳定性。【结论】本研究为解决自动驾驶多尺度目标检测难题提供了新思路,未来将优化模型结构以进一步提升效能。展开更多
In this paper, we built upon the estimating primaries by sparse inversion （EPSI） method. We use the 3D curvelet transform and modify the EPSI method to the sparse inversion of the biconvex optimization and Ll-norm r...In this paper, we built upon the estimating primaries by sparse inversion （EPSI） method. We use the 3D curvelet transform and modify the EPSI method to the sparse inversion of the biconvex optimization and Ll-norm regularization, and use alternating optimization to directly estimate the primary reflection coefficients and source wavelet. The 3D curvelet transform is used as a sparseness constraint when inverting the primary reflection coefficients, which results in avoiding the prediction subtraction process in the surface-related multiples elimination （SRME） method. The proposed method not only reduces the damage to the effective waves but also improves the elimination of multiples. It is also a wave equation- based method for elimination of surface multiple reflections, which effectively removes surface multiples under complex submarine conditions.展开更多
Three transformation models （Bursa-Wolf, Molodensky, and WTUSM） are generally used between two data systems transformation. The linear models are used when the rotation angles are small; however, when the rotation a...Three transformation models （Bursa-Wolf, Molodensky, and WTUSM） are generally used between two data systems transformation. The linear models are used when the rotation angles are small; however, when the rotation angles get bigger, model errors will be produced. In this paper, we present a method with three main terms：① the traditional rotation angles θ,φ,ψ are substituted with a,b,c which are three respective values in the anti-symmetrical or Lodrigues matrix; ② directly and accurately calculating the formula of seven parameters in any value of rotation angles; and ③ a corresponding adjustment model is established. This method does not use the triangle function. Instead it uses addition, subtraction, multiplication and division, and the complexity of the equation is reduced, making the calculation easy and quick.展开更多
目的因为有雨图像中雨线存在方向、密度和大小等各方面的差异,单幅图像去雨依旧是一个充满挑战的研究问题。现有算法在某些复杂图像上仍存在过度去雨或去雨不足等问题,部分复杂图像的边缘高频信息在去雨过程中被抹除,或图像中残留雨成...目的因为有雨图像中雨线存在方向、密度和大小等各方面的差异,单幅图像去雨依旧是一个充满挑战的研究问题。现有算法在某些复杂图像上仍存在过度去雨或去雨不足等问题,部分复杂图像的边缘高频信息在去雨过程中被抹除,或图像中残留雨成分。针对上述问题,本文提出三维注意力和Transformer去雨网络(three-dimension attention and Transformer deraining network,TDATDN)。方法将三维注意力机制与残差密集块结构相结合,以解决残差密集块通道高维度特征融合问题;使用Transformer计算特征全局关联性;针对去雨过程中图像高频信息被破坏和结构信息被抹除的问题,将多尺度结构相似性损失与常用图像去雨损失函数结合参与去雨网络训练。结果本文将提出的TDATDN网络在Rain12000雨线数据集上进行实验。其中,峰值信噪比(peak signal to noise ratio,PSNR)达到33.01 d B,结构相似性(structural similarity,SSIM)达到0.9278。实验结果表明,本文算法对比以往基于深度学习的神经网络去雨算法,显著改善了单幅图像去雨效果。结论本文提出的TDATDN图像去雨网络结合了3D注意力机制、Transformer和编码器—解码器架构的优点,可较好地完成单幅图像去雨工作。展开更多
激光雷达点云3D物体检测,对于小物体如行人、自行车的检测精度较低,容易漏检误检,提出一种多尺度Transformer激光雷达点云3D物体检测方法 MSPT-RCNN(multi-scale point transformer-RCNN),提高点云3D物体检测精度。该方法包含两个阶段,...激光雷达点云3D物体检测,对于小物体如行人、自行车的检测精度较低,容易漏检误检,提出一种多尺度Transformer激光雷达点云3D物体检测方法 MSPT-RCNN(multi-scale point transformer-RCNN),提高点云3D物体检测精度。该方法包含两个阶段,即第一阶段(RPN)和第二阶段(RCNN)。RPN阶段通过多尺度Transformer网络提取点云特征,该网络包含多尺度邻域嵌入模块和跳跃连接偏移注意力模块,获取多尺度邻域几何信息和不同层次全局语义信息,生成高质量初始3D包围盒;在RCNN阶段,引入包围盒内的点云多尺度邻域几何信息,优化了包围盒位置、尺寸、朝向和置信度等信息。实验结果表明,该方法(MSPT-RCNN)具有较高检测精度,特别是对于远处和较小物体,提升更高。MSPT-RCNN通过有效学习点云数据中的多尺度几何信息,提取不同层次有效的语义信息,能够有效提升3D物体检测精度。展开更多
When linearizing three-dimensional(3 D)coordinate similarity transformation model with large rotations,we usually encounter the ill-posed normal matrix which may aggravate the instability of solutions.To alleviate the...When linearizing three-dimensional(3 D)coordinate similarity transformation model with large rotations,we usually encounter the ill-posed normal matrix which may aggravate the instability of solutions.To alleviate the problem,a series of conversions are contributed to the 3 D coordinate similarity transformation model in this paper.We deduced a complete solution for the 3 D coordinate similarity transformation at any rotation with the nonlinear adjustment methodology,which involves the errors of the common and the non-common points.Furthermore,as the large condition number of the normal matrix resulted in an intractable form,we introduced the bary-centralization technique and a surrogate process for deterministic element of the normal matrix,and proved its benefit for alleviating the condition number.The experimental results show that our approach can obtain the smaller condition number to stabilize the convergence of the interested parameters.Especially,our approach can be implemented for considering the errors of the common and the non-common points,thus the accuracy of the transformed coordinates improves.展开更多
基金supported by the National Natural Science Foundation of China(6157206361401308)+6 种基金the Fundamental Research Funds for the Central Universities(2016YJS039)the Natural Science Foundation of Hebei Province(F2016201142F2016201187)the Natural Social Foundation of Hebei Province(HB15TQ015)the Science Research Project of Hebei Province(QN2016085ZC2016040)the Natural Science Foundation of Hebei University(2014-303)
文摘Fusion methods based on multi-scale transforms have become the mainstream of the pixel-level image fusion. However,most of these methods cannot fully exploit spatial domain information of source images, which lead to the degradation of image.This paper presents a fusion framework based on block-matching and 3D(BM3D) multi-scale transform. The algorithm first divides the image into different blocks and groups these 2D image blocks into 3D arrays by their similarity. Then it uses a 3D transform which consists of a 2D multi-scale and a 1D transform to transfer the arrays into transform coefficients, and then the obtained low-and high-coefficients are fused by different fusion rules. The final fused image is obtained from a series of fused 3D image block groups after the inverse transform by using an aggregation process. In the experimental part, we comparatively analyze some existing algorithms and the using of different transforms, e.g. non-subsampled Contourlet transform(NSCT), non-subsampled Shearlet transform(NSST), in the 3D transform step. Experimental results show that the proposed fusion framework can not only improve subjective visual effect, but also obtain better objective evaluation criteria than state-of-the-art methods.
文摘为解决基于Transformer模型的配准方法不能有效表示待配准图像的全局和局部特征、解码器无法有效还原全局上下文信息等问题,基于Swin-Transformer和UNet模型,对Swin-UNet网络架构进行扩展,增强其表示全局和局部特征的能力.并且将最新的动态蛇形卷积推广到三维空间,建立3D DSC-STUNet(Swin TransformerUNet with 3D Dynamic Snake Convolution)模型用于图像配准.相比传统卷积方法,三维动态蛇形卷积能利用可变卷积核能更好地关注图像的局部细节,提高网络处理局部特征的能力.所提出的配准方法利用特征变换提取图像特征来生成形变场,并在此基础上以无监督的方式生成配准图像.在IXI(脑部信息数据集)和OASIS(开源影像数据集)脑部MRI(核磁共振成像)图像数据集上与五种经典的配准方法相比,实验结果表明:提出的方法能够得到更好的配准结果,Dice(戴斯相似系数)值最高可以提高18.1%.
文摘当前,步态识别的主流方法常依赖堆叠卷积层来逐步扩大感受野,以融合局部特征,这种方法大多采用浅层网络,在提取步态图像的全局特征时存在一定的局限性,并缺乏对时序周期特征信息的关注。因此提出一种融合Transformer和3D卷积的深层神经网络算法(3D convolutional gait recognition network based on adaptFormer and spect-conv,3D-ASgaitNet)。首先,初始残差卷积层将二进制轮廓数据转换为浮点编码特征图,以提供密集的低级结构特征;在此基础上,光谱层通过频域和时域的联合处理增强特征提取能力,并使用伪3D残差卷积模块进一步提取高级时空特征;最后融合AdaptFormer模块,通过轻量级的下采样-上采样网络结构,以适应不同的数据分布和任务需求,提供灵活的特征变换能力。3D-ASgaitNet分别在4个公开的室内数据集(CASIA-B、OU-MVLP)、室外数据集(GREW、Gait3D)上进行,分别取得99.84%、87.83%、45.32%、72.12%的识别准确率。实验结果表明,所提出方法在CASIA-B、Gait3D数据集中的识别准确率接近SOTA性能。
文摘【背景】传统方法因静态感受野设计较难适配城市自动驾驶场景中汽车、行人及骑行者等目标的显著尺度差异,且跨尺度特征融合易引发层级干扰。【方法】针对自动驾驶场景中多类别、多尺寸目标的3D检测中跨尺度表征一致性的关键挑战,本研究提出基于均衡化感受野的3D目标检测方法VoxTNT,通过局部-全局协同注意力机制提升检测性能。在局部层面,设计了PointSetFormer模块,引入诱导集注意力模块(Induced Set Attention Block,ISAB),通过约简的交叉注意力聚合高密度点云的细粒度几何特征,突破传统体素均值池化的信息损失瓶颈;在全局层面,设计了VoxelFormerFFN模块,将非空体素抽象为超点集并实施跨体素ISAB交互,建立长程上下文依赖关系,并将全局特征学习计算负载从O(N^(2))压缩至O(M^(2))(M<<N,M为非空体素数量),规避了复杂的Transformer直接使用在原始点云造成的高计算复杂度。该双域耦合架构实现了局部细粒度感知与全局语义关联的动态平衡,有效缓解固定感受野和多尺度融合导致的特征建模偏差。【结果】实验表明,该方法在KITTI数据集单阶段检测下,中等难度级别的行人检测精度AP(Average Precision)值达到59.56%,较SECOND基线提高约12.4%,两阶段检测下以66.54%的综合指标mAP(mean Average Precision)领先次优方法BSAODet的66.10%。同时,在WOD数据集中验证了方法的有效性,综合指标mAP达到66.09%分别超越SECOND和PointPillars基线7.7%和8.5%。消融实验进一步表明,均衡化局部和全局感受野的3D特征学习机制能显著提升小目标检测精度(如在KITTI数据集中全组件消融的情况下,中等难度级别的行人和骑行者检测精度分别下降10.8%和10.0%),同时保持大目标检测的稳定性。【结论】本研究为解决自动驾驶多尺度目标检测难题提供了新思路,未来将优化模型结构以进一步提升效能。
基金supported by the National Science and Technology Major Project (No.2011ZX05023-005-008)
文摘In this paper, we built upon the estimating primaries by sparse inversion （EPSI） method. We use the 3D curvelet transform and modify the EPSI method to the sparse inversion of the biconvex optimization and Ll-norm regularization, and use alternating optimization to directly estimate the primary reflection coefficients and source wavelet. The 3D curvelet transform is used as a sparseness constraint when inverting the primary reflection coefficients, which results in avoiding the prediction subtraction process in the surface-related multiples elimination （SRME） method. The proposed method not only reduces the damage to the effective waves but also improves the elimination of multiples. It is also a wave equation- based method for elimination of surface multiple reflections, which effectively removes surface multiples under complex submarine conditions.
文摘Three transformation models （Bursa-Wolf, Molodensky, and WTUSM） are generally used between two data systems transformation. The linear models are used when the rotation angles are small; however, when the rotation angles get bigger, model errors will be produced. In this paper, we present a method with three main terms：① the traditional rotation angles θ,φ,ψ are substituted with a,b,c which are three respective values in the anti-symmetrical or Lodrigues matrix; ② directly and accurately calculating the formula of seven parameters in any value of rotation angles; and ③ a corresponding adjustment model is established. This method does not use the triangle function. Instead it uses addition, subtraction, multiplication and division, and the complexity of the equation is reduced, making the calculation easy and quick.
文摘目的因为有雨图像中雨线存在方向、密度和大小等各方面的差异,单幅图像去雨依旧是一个充满挑战的研究问题。现有算法在某些复杂图像上仍存在过度去雨或去雨不足等问题,部分复杂图像的边缘高频信息在去雨过程中被抹除,或图像中残留雨成分。针对上述问题,本文提出三维注意力和Transformer去雨网络(three-dimension attention and Transformer deraining network,TDATDN)。方法将三维注意力机制与残差密集块结构相结合,以解决残差密集块通道高维度特征融合问题;使用Transformer计算特征全局关联性;针对去雨过程中图像高频信息被破坏和结构信息被抹除的问题,将多尺度结构相似性损失与常用图像去雨损失函数结合参与去雨网络训练。结果本文将提出的TDATDN网络在Rain12000雨线数据集上进行实验。其中,峰值信噪比(peak signal to noise ratio,PSNR)达到33.01 d B,结构相似性(structural similarity,SSIM)达到0.9278。实验结果表明,本文算法对比以往基于深度学习的神经网络去雨算法,显著改善了单幅图像去雨效果。结论本文提出的TDATDN图像去雨网络结合了3D注意力机制、Transformer和编码器—解码器架构的优点,可较好地完成单幅图像去雨工作。
文摘激光雷达点云3D物体检测,对于小物体如行人、自行车的检测精度较低,容易漏检误检,提出一种多尺度Transformer激光雷达点云3D物体检测方法 MSPT-RCNN(multi-scale point transformer-RCNN),提高点云3D物体检测精度。该方法包含两个阶段,即第一阶段(RPN)和第二阶段(RCNN)。RPN阶段通过多尺度Transformer网络提取点云特征,该网络包含多尺度邻域嵌入模块和跳跃连接偏移注意力模块,获取多尺度邻域几何信息和不同层次全局语义信息,生成高质量初始3D包围盒;在RCNN阶段,引入包围盒内的点云多尺度邻域几何信息,优化了包围盒位置、尺寸、朝向和置信度等信息。实验结果表明,该方法(MSPT-RCNN)具有较高检测精度,特别是对于远处和较小物体,提升更高。MSPT-RCNN通过有效学习点云数据中的多尺度几何信息,提取不同层次有效的语义信息,能够有效提升3D物体检测精度。
基金supported by the National Natural Science Foundation of China,Nos.41874001 and 41664001Support Program for Outstanding Youth Talents in Jiangxi Province,No.20162BCB23050National Key Research and Development Program,No.2016YFB0501405。
文摘When linearizing three-dimensional(3 D)coordinate similarity transformation model with large rotations,we usually encounter the ill-posed normal matrix which may aggravate the instability of solutions.To alleviate the problem,a series of conversions are contributed to the 3 D coordinate similarity transformation model in this paper.We deduced a complete solution for the 3 D coordinate similarity transformation at any rotation with the nonlinear adjustment methodology,which involves the errors of the common and the non-common points.Furthermore,as the large condition number of the normal matrix resulted in an intractable form,we introduced the bary-centralization technique and a surrogate process for deterministic element of the normal matrix,and proved its benefit for alleviating the condition number.The experimental results show that our approach can obtain the smaller condition number to stabilize the convergence of the interested parameters.Especially,our approach can be implemented for considering the errors of the common and the non-common points,thus the accuracy of the transformed coordinates improves.
