Image fusion can be performed at different levels:signal,pixel,feature and symbol levels.Almost all image fusion algorithms developed to date fall into pixel level.This paper provides an overview of the most widely us...Image fusion can be performed at different levels:signal,pixel,feature and symbol levels.Almost all image fusion algorithms developed to date fall into pixel level.This paper provides an overview of the most widely used pixel-level image fusion algorithms and some comments about their relative strengths and weaknesses.Particular emphasis is placed on multiscale-based methods.Some performance measures practicable for pixel-level image fusion are also discussed.At last,prospects of pixel-level image fusion are made.展开更多
Infrared(IR) small target detection is one of the key technologies of infrared search and track(IRST)systems. Existing methods have some limitations in detection performance, especially when the target size is irregul...Infrared(IR) small target detection is one of the key technologies of infrared search and track(IRST)systems. Existing methods have some limitations in detection performance, especially when the target size is irregular or the background is complex. In this paper, we propose a pixel-level local contrast measure(PLLCM), which can subdivide small targets and backgrounds at pixel level simultaneously.With pixel-level segmentation, the difference between the target and the background becomes more obvious, which helps to improve the detection performance. First, we design a multiscale sliding window to quickly extract candidate target pixels. Then, a local window based on random walker(RW) is designed for pixel-level target segmentation. After that, PLLCM incorporating probability weights and scale constraints is proposed to accurately measure local contrast and suppress various types of background interference. Finally, an adaptive threshold operation is applied to separate the target from the PLLCM enhanced map. Experimental results show that the proposed method has a higher detection rate and a lower false alarm rate than the baseline algorithms, while achieving a high speed.展开更多
Onboard visual object tracking in unmanned aerial vehicles(UAVs)has attractedmuch interest due to its versatility.Meanwhile,due to high precision,Siamese networks are becoming hot spots in visual object tracking.Howev...Onboard visual object tracking in unmanned aerial vehicles(UAVs)has attractedmuch interest due to its versatility.Meanwhile,due to high precision,Siamese networks are becoming hot spots in visual object tracking.However,most Siamese trackers fail to balance the tracking accuracy and time within onboard limited computational resources of UAVs.To meet the tracking precision and real-time requirements,this paper proposes a Siamese dense pixel-level network for UAV object tracking named SiamDPL.Specifically,the Siamese network extracts features of the search region and the template region through a parameter-shared backbone network,then performs correlationmatching to obtain the candidate regionwith high similarity.To improve the matching effect of template and search features,this paper designs a dense pixel-level feature fusion module to enhance the matching ability by pixel-wise correlation and enrich the feature diversity by dense connection.An attention module composed of self-attention and channel attention is introduced to learn global context information and selectively emphasize the target feature region in the spatial and channel dimensions.In addition,a target localization module is designed to improve target location accuracy.Compared with other advanced trackers,experiments on two public benchmarks,which are UAV123@10fps and UAV20L fromthe unmanned air vehicle123(UAV123)dataset,show that SiamDPL can achieve superior performance and low complexity with a running speed of 100.1 fps on NVIDIA TITAN RTX.展开更多
Camouflaged targets are a type of nonsalient target with high foreground and background fusion and minimal target feature information,making target recognition extremely difficult.Most detection algorithms for camoufl...Camouflaged targets are a type of nonsalient target with high foreground and background fusion and minimal target feature information,making target recognition extremely difficult.Most detection algorithms for camouflaged targets use only the target’s single-band information,resulting in low detection accuracy and a high missed detection rate.We present a multimodal image fusion camouflaged target detection technique (MIF-YOLOv5) in this paper.First,we provide a multimodal image input to achieve pixel-level fusion of the camouflaged target’s optical and infrared images to improve the effective feature information of the camouflaged target.Second,a loss function is created,and the K-Means++clustering technique is used to optimize the target anchor frame in the dataset to increase camouflage personnel detection accuracy and robustness.Finally,a comprehensive detection index of camouflaged targets is proposed to compare the overall effectiveness of various approaches.More crucially,we create a multispectral camouflage target dataset to test the suggested technique.Experimental results show that the proposed method has the best comprehensive detection performance,with a detection accuracy of 96.5%,a recognition probability of92.5%,a parameter number increase of 1×10^(4),a theoretical calculation amount increase of 0.03 GFLOPs,and a comprehensive detection index of 0.85.The advantage of this method in terms of detection accuracy is also apparent in performance comparisons with other target algorithms.展开更多
现有的异常检测方法能在特定应用场景下实现高精度检测,然而这些方法难以适用于其他应用场景,且自动化程度有限。因此,提出一种视觉基础模型(VFM)驱动的像素级图像异常检测方法SSMOD-Net(State Space Model driven-Omni Dimensional Ne...现有的异常检测方法能在特定应用场景下实现高精度检测,然而这些方法难以适用于其他应用场景,且自动化程度有限。因此,提出一种视觉基础模型(VFM)驱动的像素级图像异常检测方法SSMOD-Net(State Space Model driven-Omni Dimensional Net),旨在实现更精确的工业缺陷检测。与现有方法不同,SSMOD-Net实现SAM(Segment Anything Model)的自动化提示且不需要微调SAM,因此特别适用于需要处理大规模工业视觉数据的场景。SSMOD-Net的核心是一个新颖的提示编码器,该编码器由状态空间模型驱动,能够根据SAM的输入图像动态地生成提示。这一设计允许模型在保持SAM架构不变的同时,通过提示编码器引入额外的指导信息,从而提高检测精度。提示编码器内部集成一个残差多尺度模块,该模块基于状态空间模型构建,能够综合利用多尺度信息和全局信息。这一模块通过迭代搜索,在提示空间中寻找最优的提示,并将这些提示以高维张量的形式提供给SAM,从而增强模型对工业异常的识别能力。而且所提方法不需要对SAM进行任何修改,从而避免复杂的对训练计划的微调需求。在多个数据集上的实验结果表明,所提方法展现出了卓越的性能,与AutoSAM和SAM-EG(SAM with Edge Guidance framework for efficient polyp segmentation)等方法相比,所提方法在mE(mean E-measure)和平均绝对误差(MAE)、Dice和交并比(IoU)上都取得了较好的结果。展开更多
This paper presents a 50 Hz 15-bit analog-to-digital converter （ADC） for pixel-level implementation in CMOS image sensors. The ADC is based on charge packets counting and adopts a voltage reset technique to inject c...This paper presents a 50 Hz 15-bit analog-to-digital converter （ADC） for pixel-level implementation in CMOS image sensors. The ADC is based on charge packets counting and adopts a voltage reset technique to inject charge packets. The core circuit for charge/pulse conversion is specially optimized for low power, low noise and small area. An experimental chip with ten pixel-level ADCs has been fabricated and tested for verification. The measurement result shows a standard deviation of 1.8 LSB for full-scale output. The ADC has an area of 45 × 45μm^2 and consumes less than 2 μW in a standard 1P-6M 0.18μm CMOS process.展开更多
Accurate global land cover(GLC), as a key input for scientific communities, is important for a wide variety of applications. In order to understand the current suitability and limitation of GLC products, the discrepan...Accurate global land cover(GLC), as a key input for scientific communities, is important for a wide variety of applications. In order to understand the current suitability and limitation of GLC products, the discrepancy and pixellevel uncertainty in major GLC products in three epochs are assessed in this study by using an integrated uncertainty index(IUI) that combines the thematic uncertainty and local classification accuracy uncertainty. The results show that the overall spatial agreements(Ao values) between GLC products are lower than 58%, and the total areas of forests are very consistent in major GLC products, but significant differences are found in different forest classes.The misclassification among different forest classes and mosaic types can account for about 20% of the total disagreements. The mean IUI almost reaches 0.5, and high uncertainty mostly occurs in transition zones and heterogeneous areas across the world. Further efforts are needed to make in the land cover classifications in areas with high uncertainty. Designing a classification scheme for climate models, with explicit definitions of land cover classes in the threshold of common attributes, is urgently needed. Information of the pixel-level uncertainty in major GLC products not only give important implications for the specific application, but also provide a quite important basis for land cover fusion.展开更多
电子轰击型有源像素传感器(Electron Bombarded Active Pixel Sensor,EBAPS)作为一种数字化微光成像器件,以其成像系统具有小型化、低成本和低功耗的优势,以及在昼夜条件下连续拍摄时具备清晰成像的能力,成为微光成像领域的研究热点。...电子轰击型有源像素传感器(Electron Bombarded Active Pixel Sensor,EBAPS)作为一种数字化微光成像器件,以其成像系统具有小型化、低成本和低功耗的优势,以及在昼夜条件下连续拍摄时具备清晰成像的能力,成为微光成像领域的研究热点。文章基于国产某型EBAPS微光器件,利用FPGA作为核心处理器,完成了EBAPS器件驱动电路、图像处理和跟踪电路、显示电路的设计。同时,构建了满足昼夜复用的光学系统,搭建了一种小型化的手持成像跟踪系统。实验结果表明,该EBAPS昼夜成像系统可在1×10^(-4)~1×10^(4) lx照度条件下实现良好的成像和跟踪效果。展开更多
基金the National Natural Science Foundation of China (Nos. 60775022 and 60705006)
文摘Image fusion can be performed at different levels:signal,pixel,feature and symbol levels.Almost all image fusion algorithms developed to date fall into pixel level.This paper provides an overview of the most widely used pixel-level image fusion algorithms and some comments about their relative strengths and weaknesses.Particular emphasis is placed on multiscale-based methods.Some performance measures practicable for pixel-level image fusion are also discussed.At last,prospects of pixel-level image fusion are made.
基金supported by the National Natural Science Foundation of China under Grant 62003247, Grant 62075169, and Grant 62061160370。
文摘Infrared(IR) small target detection is one of the key technologies of infrared search and track(IRST)systems. Existing methods have some limitations in detection performance, especially when the target size is irregular or the background is complex. In this paper, we propose a pixel-level local contrast measure(PLLCM), which can subdivide small targets and backgrounds at pixel level simultaneously.With pixel-level segmentation, the difference between the target and the background becomes more obvious, which helps to improve the detection performance. First, we design a multiscale sliding window to quickly extract candidate target pixels. Then, a local window based on random walker(RW) is designed for pixel-level target segmentation. After that, PLLCM incorporating probability weights and scale constraints is proposed to accurately measure local contrast and suppress various types of background interference. Finally, an adaptive threshold operation is applied to separate the target from the PLLCM enhanced map. Experimental results show that the proposed method has a higher detection rate and a lower false alarm rate than the baseline algorithms, while achieving a high speed.
基金funded by the National Natural Science Foundation of China(Grant No.52072408),author Y.C.
文摘Onboard visual object tracking in unmanned aerial vehicles(UAVs)has attractedmuch interest due to its versatility.Meanwhile,due to high precision,Siamese networks are becoming hot spots in visual object tracking.However,most Siamese trackers fail to balance the tracking accuracy and time within onboard limited computational resources of UAVs.To meet the tracking precision and real-time requirements,this paper proposes a Siamese dense pixel-level network for UAV object tracking named SiamDPL.Specifically,the Siamese network extracts features of the search region and the template region through a parameter-shared backbone network,then performs correlationmatching to obtain the candidate regionwith high similarity.To improve the matching effect of template and search features,this paper designs a dense pixel-level feature fusion module to enhance the matching ability by pixel-wise correlation and enrich the feature diversity by dense connection.An attention module composed of self-attention and channel attention is introduced to learn global context information and selectively emphasize the target feature region in the spatial and channel dimensions.In addition,a target localization module is designed to improve target location accuracy.Compared with other advanced trackers,experiments on two public benchmarks,which are UAV123@10fps and UAV20L fromthe unmanned air vehicle123(UAV123)dataset,show that SiamDPL can achieve superior performance and low complexity with a running speed of 100.1 fps on NVIDIA TITAN RTX.
基金Project supported by the Shandong Provincial Natural Science Foundation of China(No.ZR2020MF015)the Aerospace Science and Technology Innovation Institute Stabilization Support Project(No.ZY0110020009)。
文摘Camouflaged targets are a type of nonsalient target with high foreground and background fusion and minimal target feature information,making target recognition extremely difficult.Most detection algorithms for camouflaged targets use only the target’s single-band information,resulting in low detection accuracy and a high missed detection rate.We present a multimodal image fusion camouflaged target detection technique (MIF-YOLOv5) in this paper.First,we provide a multimodal image input to achieve pixel-level fusion of the camouflaged target’s optical and infrared images to improve the effective feature information of the camouflaged target.Second,a loss function is created,and the K-Means++clustering technique is used to optimize the target anchor frame in the dataset to increase camouflage personnel detection accuracy and robustness.Finally,a comprehensive detection index of camouflaged targets is proposed to compare the overall effectiveness of various approaches.More crucially,we create a multispectral camouflage target dataset to test the suggested technique.Experimental results show that the proposed method has the best comprehensive detection performance,with a detection accuracy of 96.5%,a recognition probability of92.5%,a parameter number increase of 1×10^(4),a theoretical calculation amount increase of 0.03 GFLOPs,and a comprehensive detection index of 0.85.The advantage of this method in terms of detection accuracy is also apparent in performance comparisons with other target algorithms.
文摘现有的异常检测方法能在特定应用场景下实现高精度检测,然而这些方法难以适用于其他应用场景,且自动化程度有限。因此,提出一种视觉基础模型(VFM)驱动的像素级图像异常检测方法SSMOD-Net(State Space Model driven-Omni Dimensional Net),旨在实现更精确的工业缺陷检测。与现有方法不同,SSMOD-Net实现SAM(Segment Anything Model)的自动化提示且不需要微调SAM,因此特别适用于需要处理大规模工业视觉数据的场景。SSMOD-Net的核心是一个新颖的提示编码器,该编码器由状态空间模型驱动,能够根据SAM的输入图像动态地生成提示。这一设计允许模型在保持SAM架构不变的同时,通过提示编码器引入额外的指导信息,从而提高检测精度。提示编码器内部集成一个残差多尺度模块,该模块基于状态空间模型构建,能够综合利用多尺度信息和全局信息。这一模块通过迭代搜索,在提示空间中寻找最优的提示,并将这些提示以高维张量的形式提供给SAM,从而增强模型对工业异常的识别能力。而且所提方法不需要对SAM进行任何修改,从而避免复杂的对训练计划的微调需求。在多个数据集上的实验结果表明,所提方法展现出了卓越的性能,与AutoSAM和SAM-EG(SAM with Edge Guidance framework for efficient polyp segmentation)等方法相比,所提方法在mE(mean E-measure)和平均绝对误差(MAE)、Dice和交并比(IoU)上都取得了较好的结果。
基金supported by the Major National Science & Technology Program of China(No.2012ZX03004004-002)
文摘This paper presents a 50 Hz 15-bit analog-to-digital converter （ADC） for pixel-level implementation in CMOS image sensors. The ADC is based on charge packets counting and adopts a voltage reset technique to inject charge packets. The core circuit for charge/pulse conversion is specially optimized for low power, low noise and small area. An experimental chip with ten pixel-level ADCs has been fabricated and tested for verification. The measurement result shows a standard deviation of 1.8 LSB for full-scale output. The ADC has an area of 45 × 45μm^2 and consumes less than 2 μW in a standard 1P-6M 0.18μm CMOS process.
基金Supported by the National Key Research and Development Program of China(2016YFA0600303 and 2018YFC1506506)。
文摘Accurate global land cover(GLC), as a key input for scientific communities, is important for a wide variety of applications. In order to understand the current suitability and limitation of GLC products, the discrepancy and pixellevel uncertainty in major GLC products in three epochs are assessed in this study by using an integrated uncertainty index(IUI) that combines the thematic uncertainty and local classification accuracy uncertainty. The results show that the overall spatial agreements(Ao values) between GLC products are lower than 58%, and the total areas of forests are very consistent in major GLC products, but significant differences are found in different forest classes.The misclassification among different forest classes and mosaic types can account for about 20% of the total disagreements. The mean IUI almost reaches 0.5, and high uncertainty mostly occurs in transition zones and heterogeneous areas across the world. Further efforts are needed to make in the land cover classifications in areas with high uncertainty. Designing a classification scheme for climate models, with explicit definitions of land cover classes in the threshold of common attributes, is urgently needed. Information of the pixel-level uncertainty in major GLC products not only give important implications for the specific application, but also provide a quite important basis for land cover fusion.
