Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the...Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.展开更多
Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targ...Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.展开更多
In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS...In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.展开更多
Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides ...Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.展开更多
Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in ...Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in turbidite and other debrite deposits. By high-resolution imaging, three sedimentary processes were distinguished with distinct acoustic features. 1. Slumps and slides occur with contrasting backscatter, rough surface textures, blockings, and acoustic shadows at headwalls. They are very extensive and often in lobate form downslope. 2. Debris flow has uniform, general medium backscatter, sometimes showing marbling/lineation in lobate form. 3. Turbidity current is characterized by low backscatter confined to the channel as acoustic signal is attenuated. Regional tectonics must be the dominating factor that controls deposition pattern in this area.展开更多
Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging alo...Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.展开更多
Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated...Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.展开更多
基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型...基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型的侧扫声呐图像海底地形重建方法。首先在保留Lambert漫反射模型的基础上引入Blinn-Phong镜面反射模型,并基于侧扫声呐图像粗糙度进行漫反射系数、镜面反射系数和镜面反射指数的自适应设计,从而形成适合复杂海底表面的混合反射模型;然后线性化处理辐射照度方程,并采用牛顿-拉夫逊方法来获得海底表面高程;最后根据海底点设计反演高程约束系数来约束表面高程,进而完成侧扫声呐图像的三维重建。试验测试结果表明,相较于其他三维重建方法,文中方法的平均绝对误差值VMAE平均降低了32.18%、相关系数值VCC平均提高了29.62%、信噪比值VSNR平均提升了27.23%,有效实现了侧扫声呐图像三维重建。展开更多
明暗恢复形状(shape from shading,SFS)算法是三维重建中不可缺少的关键技术。该方法通过单幅图像中物体的灰度的亮度信息,求出每个点的相对高度和表面法向量,现有的SFS线性化算法利用泰勒级数进行展开忽略了高阶部分的影响,导致三维重...明暗恢复形状(shape from shading,SFS)算法是三维重建中不可缺少的关键技术。该方法通过单幅图像中物体的灰度的亮度信息,求出每个点的相对高度和表面法向量,现有的SFS线性化算法利用泰勒级数进行展开忽略了高阶部分的影响,导致三维重建的图像不够精确等问题。为解决三维重建精度问题,本文提出一种基于SFS算法改进的高精度快速水下图像三维重建方法,在提高三维重建精度的前提下,解决了对本轮数据利用率不够、迭代速度慢等问题。将其应用在真实图像和虚拟图像上进行实验评估以及三维重建的量化分析,验证了相比于经典SFS线性化算法和SFS最小化算法,本文提出的SFS算法改进的高精度快速水下图像三维重建方法,取得了令人满意的信噪比和图像信息熵值。展开更多
基金supported in part by the Hainan Provincial Natural Science Foundation(Grant No.420CXTD439)Sanya Science and Technology Special Fund(Grant No.2022KJCX83)+1 种基金Institute and Local Cooperation Foundation of Sanya in China(Grant No.2019YD08)National Natural Science Foundation of China(Grant No.61661038).
文摘Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2024-00334159)the Korea Institute of Ocean Science and Technology(KIOST)project entitled“Development of Maritime Domain Awareness Technology for Sea Power Enhancement”(PEA0332).
文摘Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.
基金This work is partially supported by the Natural Key Research and Development Program of China(No.2016YF C0301400).
文摘In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.
基金funded by the Natural Science Foundation of Fujian Province(No.2018J01063)the Project of Deep Learning Based Underwater Cultural Relics Recognization(No.38360041)the Project of the State Administration of Cultural Relics(No.2018300).
文摘Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.
基金Financially supported by the NSFC (Grant No.40276022), KnowledgeInnovation Program of Chinese Academy of Sciences (KZCX3-SW-219)and JSPS international cooperation program, and the Ministry of Scienceand Technology Project (G200046704)
文摘Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in turbidite and other debrite deposits. By high-resolution imaging, three sedimentary processes were distinguished with distinct acoustic features. 1. Slumps and slides occur with contrasting backscatter, rough surface textures, blockings, and acoustic shadows at headwalls. They are very extensive and often in lobate form downslope. 2. Debris flow has uniform, general medium backscatter, sometimes showing marbling/lineation in lobate form. 3. Turbidity current is characterized by low backscatter confined to the channel as acoustic signal is attenuated. Regional tectonics must be the dominating factor that controls deposition pattern in this area.
基金supported by the National Key R&D Program of China(Grant No.2023YFC3010803)the National Nature Science Foundation of China(Grant No.52272424)+1 种基金the Key R&D Program of Hubei Province of China(Grant No.2023BCB123)the Fundamental Research Funds for the Central Universities(Grant No.WUT:2023IVB079)。
文摘Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.
文摘Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.
文摘基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型的侧扫声呐图像海底地形重建方法。首先在保留Lambert漫反射模型的基础上引入Blinn-Phong镜面反射模型,并基于侧扫声呐图像粗糙度进行漫反射系数、镜面反射系数和镜面反射指数的自适应设计,从而形成适合复杂海底表面的混合反射模型;然后线性化处理辐射照度方程,并采用牛顿-拉夫逊方法来获得海底表面高程;最后根据海底点设计反演高程约束系数来约束表面高程,进而完成侧扫声呐图像的三维重建。试验测试结果表明,相较于其他三维重建方法,文中方法的平均绝对误差值VMAE平均降低了32.18%、相关系数值VCC平均提高了29.62%、信噪比值VSNR平均提升了27.23%,有效实现了侧扫声呐图像三维重建。
文摘明暗恢复形状(shape from shading,SFS)算法是三维重建中不可缺少的关键技术。该方法通过单幅图像中物体的灰度的亮度信息,求出每个点的相对高度和表面法向量,现有的SFS线性化算法利用泰勒级数进行展开忽略了高阶部分的影响,导致三维重建的图像不够精确等问题。为解决三维重建精度问题,本文提出一种基于SFS算法改进的高精度快速水下图像三维重建方法,在提高三维重建精度的前提下,解决了对本轮数据利用率不够、迭代速度慢等问题。将其应用在真实图像和虚拟图像上进行实验评估以及三维重建的量化分析,验证了相比于经典SFS线性化算法和SFS最小化算法,本文提出的SFS算法改进的高精度快速水下图像三维重建方法,取得了令人满意的信噪比和图像信息熵值。
