Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the...Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.展开更多
Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging alo...Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.展开更多
In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS...In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.展开更多
Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides ...Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.展开更多
Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated...Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.展开更多
Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targ...Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.展开更多
Underwater target detection in forward-looking sonar(FLS)images is a challenging but promising endeavor.The existing neural-based methods yield notable progress but there remains room for improvement due to overlookin...Underwater target detection in forward-looking sonar(FLS)images is a challenging but promising endeavor.The existing neural-based methods yield notable progress but there remains room for improvement due to overlooking the unique characteristics of underwater environments.Considering the problems of low imaging resolution,complex background environment,and large changes in target imaging of underwater sonar images,this paper specifically designs a sonar images target detection Network based on Progressive sensitivity capture,named ProNet.It progressively captures the sensitive regions in the current image where potential effective targets may exist.Guided by this basic idea,the primary technical innovation of this paper is the introduction of a foundational module structure for constructing a sonar target detection backbone network.This structure employs a multi-subspace mixed convolution module that initially maps sonar images into different subspaces and extracts local contextual features using varying convolutional receptive fields within these heterogeneous subspaces.Subsequently,a Scale-aware aggregation module effectively aggregates the heterogeneous features extracted from different subspaces.Finally,the multi-scale attention structure further enhances the relational perception of the aggregated features.We evaluated ProNet on three FLS datasets of varying scenes,and experimental results indicate that ProNet outperforms the current state-of-the-art sonar image and general target detectors.展开更多
The risk of gas leakage due to geological flaws in offshore carbon capture, utilization, and storage, as well as leakage from underwater oil or gas pipelines, highlights the need for underwater gas leakage monitoring ...The risk of gas leakage due to geological flaws in offshore carbon capture, utilization, and storage, as well as leakage from underwater oil or gas pipelines, highlights the need for underwater gas leakage monitoring technology. Remotely operated vehicles(ROVs) and autonomous underwater vehicles(AUVs) are equipped with high-resolution imaging sonar systems that have broad application potential in underwater gas and target detection tasks. However, some bubble clusters are relatively weak scatterers, so detecting and distinguishing them against the seabed reverberation in forward-looking sonar images are challenging. This study uses the dual-tree complex wavelet transform to extract the image features of multibeam forward-looking sonar. Underwater gas leakages with different flows are classified by combining deep learning theory. A pool experiment is designed to simulate gas leakage, where sonar images are obtained for further processing. Results demonstrate that this method can detect and classify underwater gas leakage streams with high classification accuracy. This performance indicates that the method can detect gas leakage from multibeam forward-looking sonar images and has the potential to predict gas leakage flow.展开更多
This article introduces an underwater robot inspection anomaly localization feedback system comprising a real-time water surface tracking,detection,and positioning system located on the water surface,while the underwa...This article introduces an underwater robot inspection anomaly localization feedback system comprising a real-time water surface tracking,detection,and positioning system located on the water surface,while the underwater robot inspection anomaly feedback system is housed within the underwater robot.The system facilitates the issuance of corresponding mechanical responses based on the water surface’s real-time tracking,detection,and positioning,enabling recognition and feedback of anomaly information.Through sonar technology,the underwater robot inspection anomaly feedback system monitors the underwater robot in real-time,triggering responsive actions upon encountering anomalies.The real-time tracking,detection,and positioning system from the water surface identifies abnormal conditions of underwater robots based on changes in sonar images,subsequently notifying personnel for necessary intervention.展开更多
基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型...基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型的侧扫声呐图像海底地形重建方法。首先在保留Lambert漫反射模型的基础上引入Blinn-Phong镜面反射模型,并基于侧扫声呐图像粗糙度进行漫反射系数、镜面反射系数和镜面反射指数的自适应设计,从而形成适合复杂海底表面的混合反射模型;然后线性化处理辐射照度方程,并采用牛顿-拉夫逊方法来获得海底表面高程;最后根据海底点设计反演高程约束系数来约束表面高程,进而完成侧扫声呐图像的三维重建。试验测试结果表明,相较于其他三维重建方法,文中方法的平均绝对误差值VMAE平均降低了32.18%、相关系数值VCC平均提高了29.62%、信噪比值VSNR平均提升了27.23%,有效实现了侧扫声呐图像三维重建。展开更多
基金supported in part by the Hainan Provincial Natural Science Foundation(Grant No.420CXTD439)Sanya Science and Technology Special Fund(Grant No.2022KJCX83)+1 种基金Institute and Local Cooperation Foundation of Sanya in China(Grant No.2019YD08)National Natural Science Foundation of China(Grant No.61661038).
文摘Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.
基金supported by the National Key R&D Program of China(Grant No.2023YFC3010803)the National Nature Science Foundation of China(Grant No.52272424)+1 种基金the Key R&D Program of Hubei Province of China(Grant No.2023BCB123)the Fundamental Research Funds for the Central Universities(Grant No.WUT:2023IVB079)。
文摘Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.
基金This work is partially supported by the Natural Key Research and Development Program of China(No.2016YF C0301400).
文摘In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.
基金funded by the Natural Science Foundation of Fujian Province(No.2018J01063)the Project of Deep Learning Based Underwater Cultural Relics Recognization(No.38360041)the Project of the State Administration of Cultural Relics(No.2018300).
文摘Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.
文摘Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2024-00334159)the Korea Institute of Ocean Science and Technology(KIOST)project entitled“Development of Maritime Domain Awareness Technology for Sea Power Enhancement”(PEA0332).
文摘Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.
基金supported in part by Youth Innovation Promotion Association,Chinese Academy of Sciences under Grant 2022022in part by South China Sea Nova project of Hainan Province under Grant NHXXRCXM202340in part by the Scientific Research Foundation Project of Hainan Acoustics Laboratory under grant ZKNZ2024001.
文摘Underwater target detection in forward-looking sonar(FLS)images is a challenging but promising endeavor.The existing neural-based methods yield notable progress but there remains room for improvement due to overlooking the unique characteristics of underwater environments.Considering the problems of low imaging resolution,complex background environment,and large changes in target imaging of underwater sonar images,this paper specifically designs a sonar images target detection Network based on Progressive sensitivity capture,named ProNet.It progressively captures the sensitive regions in the current image where potential effective targets may exist.Guided by this basic idea,the primary technical innovation of this paper is the introduction of a foundational module structure for constructing a sonar target detection backbone network.This structure employs a multi-subspace mixed convolution module that initially maps sonar images into different subspaces and extracts local contextual features using varying convolutional receptive fields within these heterogeneous subspaces.Subsequently,a Scale-aware aggregation module effectively aggregates the heterogeneous features extracted from different subspaces.Finally,the multi-scale attention structure further enhances the relational perception of the aggregated features.We evaluated ProNet on three FLS datasets of varying scenes,and experimental results indicate that ProNet outperforms the current state-of-the-art sonar image and general target detectors.
文摘The risk of gas leakage due to geological flaws in offshore carbon capture, utilization, and storage, as well as leakage from underwater oil or gas pipelines, highlights the need for underwater gas leakage monitoring technology. Remotely operated vehicles(ROVs) and autonomous underwater vehicles(AUVs) are equipped with high-resolution imaging sonar systems that have broad application potential in underwater gas and target detection tasks. However, some bubble clusters are relatively weak scatterers, so detecting and distinguishing them against the seabed reverberation in forward-looking sonar images are challenging. This study uses the dual-tree complex wavelet transform to extract the image features of multibeam forward-looking sonar. Underwater gas leakages with different flows are classified by combining deep learning theory. A pool experiment is designed to simulate gas leakage, where sonar images are obtained for further processing. Results demonstrate that this method can detect and classify underwater gas leakage streams with high classification accuracy. This performance indicates that the method can detect gas leakage from multibeam forward-looking sonar images and has the potential to predict gas leakage flow.
文摘This article introduces an underwater robot inspection anomaly localization feedback system comprising a real-time water surface tracking,detection,and positioning system located on the water surface,while the underwater robot inspection anomaly feedback system is housed within the underwater robot.The system facilitates the issuance of corresponding mechanical responses based on the water surface’s real-time tracking,detection,and positioning,enabling recognition and feedback of anomaly information.Through sonar technology,the underwater robot inspection anomaly feedback system monitors the underwater robot in real-time,triggering responsive actions upon encountering anomalies.The real-time tracking,detection,and positioning system from the water surface identifies abnormal conditions of underwater robots based on changes in sonar images,subsequently notifying personnel for necessary intervention.
文摘基于明暗恢复形状(shape from shading,简称SFS)方法进行海底侧扫声呐图像三维重建时,会因为方法中反射模型与海底表面不相符而降低重建精度。为此文章在分析大陆架海底表面特点及侧扫声呐工作机理的基础上,提出了基于SFS混合反射模型的侧扫声呐图像海底地形重建方法。首先在保留Lambert漫反射模型的基础上引入Blinn-Phong镜面反射模型,并基于侧扫声呐图像粗糙度进行漫反射系数、镜面反射系数和镜面反射指数的自适应设计,从而形成适合复杂海底表面的混合反射模型;然后线性化处理辐射照度方程,并采用牛顿-拉夫逊方法来获得海底表面高程;最后根据海底点设计反演高程约束系数来约束表面高程,进而完成侧扫声呐图像的三维重建。试验测试结果表明,相较于其他三维重建方法,文中方法的平均绝对误差值VMAE平均降低了32.18%、相关系数值VCC平均提高了29.62%、信噪比值VSNR平均提升了27.23%,有效实现了侧扫声呐图像三维重建。
