Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the...Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.展开更多
Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targ...Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.展开更多
Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, fo...Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, for example, sway yaw and surge that are the most important error sources. The phase error of a wide band synthetic aperture sonar is modeled and solutions to sway yaw and surge motion estimation based on the raw sonar echo data with a Displaced Phase Center Antenna (DPCA) method are proposed and their implementations are detailed in this paper. It is shown that the sway estimates can be obtained from the correlation lag and phase difference between the returns at coincident phase centers. An estimate of yaw is also possible if such a technique is applied to more than one overlapping phase center positions. Surge estimates can be obtained by identifying pairs of phase centers with a maximum correlation coefficient. The method works only if the platform velocity is low enough such that a number of phase centers from adjacent pings overlap.展开更多
In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS...In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.展开更多
Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides ...Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.展开更多
Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging alo...Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.展开更多
Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated...Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.展开更多
A multi-beam chirp sonar based on IP connections and DSP processing nodes was proposed and designed to provide an expandable system with high-speed processing and mass-storage of real-time signals for multi-beam profi...A multi-beam chirp sonar based on IP connections and DSP processing nodes was proposed and designed to provide an expandable system with high-speed processing and mass-storage of real-time signals for multi-beam profiling sonar.The system was designed for seabed petroleum pipeline detection and orientation,and can receive echo signals and process the data in real time,refreshing the display 10 times per second.Every node of the chirp sonar connects with data processing nodes through TCP/IP. Merely by adding nodes,the system’s processing ability can be increased proportionately without changing the software.System debugging and experimental testing proved the system to be practical and stable.This design provides a new method for high speed active sonar.展开更多
Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in ...Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in turbidite and other debrite deposits. By high-resolution imaging, three sedimentary processes were distinguished with distinct acoustic features. 1. Slumps and slides occur with contrasting backscatter, rough surface textures, blockings, and acoustic shadows at headwalls. They are very extensive and often in lobate form downslope. 2. Debris flow has uniform, general medium backscatter, sometimes showing marbling/lineation in lobate form. 3. Turbidity current is characterized by low backscatter confined to the channel as acoustic signal is attenuated. Regional tectonics must be the dominating factor that controls deposition pattern in this area.展开更多
基金supported in part by the Hainan Provincial Natural Science Foundation(Grant No.420CXTD439)Sanya Science and Technology Special Fund(Grant No.2022KJCX83)+1 种基金Institute and Local Cooperation Foundation of Sanya in China(Grant No.2019YD08)National Natural Science Foundation of China(Grant No.61661038).
文摘Underwater shipwreck identification technology, as a crucial technique in the field of marine surveying, plays a significant role in areas such as the search and rescue of maritime disaster shipwrecks. When facing the task of object detection in shipwreck side-scan sonar images, due to the complex seabed environment, it is difficult to extract object features, often leading to missed detections of shipwreck images and slow detection speed. To address these issues, this paper proposes an object detection algorithm, CSC-YOLO (Context Guided Block, Shared Conv_Group Normalization Detection, Cross Stage Partial with 2 Partial Convolution-You Only Look Once), based on YOLOv8n for shipwreck side-scan sonar images. Firstly, to tackle the problem of small samples in shipwreck side-scan sonar images, a new dataset was constructed through offline data augmentation to expand data and intuitively enhance sample diversity, with the Mosaic algorithm integrated to strengthen the network’s generalization to the dataset. Subsequently, the Context Guided Block (CGB) module was introduced into the backbone network model to enhance the network’s ability to learn and express image features. Additionally, by employing Group Normalization (GN) techniques and shared convolution operations, we constructed the Shared Conv_GN Detection (SCGD) head, which improves the localization and classification performance of the detection head while significantly reducing the number of parameters and computational load. Finally, the Partial Convolution (PConv) was introduced and the Cross Stage Partial with 2 PConv (C2PC) module was constructed to help the network maintain effective extraction of spatial features while reducing computational complexity. The improved CSC-YOLO model, compared with the YOLOv8n model on the validation set, mean Average Precision (mAP) increases by 3.1%, Recall (R) increases by 6.4%, and the F1-measure (F1) increases by 4.7%. Furthermore, in the improved algorithm, the number of parameters decreases by 20%, the computational complexity decreases by 23.2%, and Frames Per Second (FPS) increases by 17.6%. In addition, compared with the advanced popular model, the superiority of the proposed model is proved. The subsequent experiments on real side-scan sonar images of shipwrecks fully demonstrate that the CSC-YOLO algorithm meets the requirements for actual side-scan sonar detection of underwater shipwrecks.
基金supported by the National Research Foundation of Korea(NRF)grant funded by the Korea government(MSIT)(No.RS-2024-00334159)the Korea Institute of Ocean Science and Technology(KIOST)project entitled“Development of Maritime Domain Awareness Technology for Sea Power Enhancement”(PEA0332).
文摘Side-scan sonar(SSS)is essential for acquiring high-resolution seafloor images over large areas,facilitat-ing the identification of subsea objects.However,military security restrictions and the scarcity of subsea targets limit the availability of SSS data,posing challenges for Automatic Target Recognition(ATR)research.This paper presents an approach that uses Cycle-Consistent Generative Adversarial Networks(CycleGAN)to augment SSS images of key subsea objects,such as shipwrecks,aircraft,and drowning victims.The process begins by constructing 3D models to generate rendered images with realistic shadows frommultiple angles.To enhance image quality,a shadowextractor and shadow region loss function are introduced to ensure consistent shadowrepresentation.Additionally,amulti-resolution learning structure enables effective training,even with limited data availability.The experimental results show that the generated data improved object detection accuracy when they were used for training and demonstrated the ability to generate clear shadow and background regions with stability.
文摘Phase errors in synthetic aperture sonar (SAS) imaging must be reduced to less than one eighth of a wavelength so as to avoid image destruction. Most of the phase errors occur as a result of platform motion errors, for example, sway yaw and surge that are the most important error sources. The phase error of a wide band synthetic aperture sonar is modeled and solutions to sway yaw and surge motion estimation based on the raw sonar echo data with a Displaced Phase Center Antenna (DPCA) method are proposed and their implementations are detailed in this paper. It is shown that the sway estimates can be obtained from the correlation lag and phase difference between the returns at coincident phase centers. An estimate of yaw is also possible if such a technique is applied to more than one overlapping phase center positions. Surge estimates can be obtained by identifying pairs of phase centers with a maximum correlation coefficient. The method works only if the platform velocity is low enough such that a number of phase centers from adjacent pings overlap.
基金This work is partially supported by the Natural Key Research and Development Program of China(No.2016YF C0301400).
文摘In ocean explorations,side-scan sonar(SSS)plays a very important role and can quickly depict seabed topography.As-sembling the SSS to an autonomous underwater vehicle(AUV)and performing semantic segmentation of an SSS image in real time can realize online submarine geomorphology or target recognition,which is conducive to submarine detection.However,because of the complexity of the marine environment,various noises in the ocean pollute the sonar image,which also encounters the intensity inhomogeneity problem.In this paper,we propose a novel neural network architecture named dilated convolutional neural network(DcNet)that can run in real time while addressing the above-mentioned issues and providing accurate semantic segmentation.The proposed architecture presents an encoder-decoder network to gradually reduce the spatial dimension of the input image and recover the details of the target,respectively.The core of our network is a novel block connection named DCblock,which mainly uses dilated convolution and depthwise separable convolution between the encoder and decoder to attain more context while still retaining high accuracy.Furthermore,our proposed method performs a super-resolution reconstruction to enlarge the dataset with high-quality im-ages.We compared our network to other common semantic segmentation networks performed on an NVIDIA Jetson TX2 using our sonar image datasets.Experimental results show that while the inference speed of the proposed network significantly outperforms state-of-the-art architectures,the accuracy of our method is still comparable,which indicates its potential applications not only in AUVs equipped with SSS but also in marine exploration.
基金funded by the Natural Science Foundation of Fujian Province(No.2018J01063)the Project of Deep Learning Based Underwater Cultural Relics Recognization(No.38360041)the Project of the State Administration of Cultural Relics(No.2018300).
文摘Seabed sediment recognition is vital for the exploitation of marine resources.Side-scan sonar(SSS)is an excellent tool for acquiring the imagery of seafloor topography.Combined with ocean surface sampling,it provides detailed and accurate images of marine substrate features.Most of the processing of SSS imagery works around limited sampling stations and requires manual interpretation to complete the classification of seabed sediment imagery.In complex sea areas,with manual interpretation,small targets are often lost due to a large amount of information.To date,studies related to the automatic recognition of seabed sediments are still few.This paper proposes a seabed sediment recognition method based on You Only Look Once version 5 and SSS imagery to perform real-time sedi-ment classification and localization for accuracy,particularly on small targets and faster speeds.We used methods such as changing the dataset size,epoch,and optimizer and adding multiscale training to overcome the challenges of having a small sample and a low accuracy.With these methods,we improved the results on mean average precision by 8.98%and F1 score by 11.12%compared with the original method.In addition,the detection speed was approximately 100 frames per second,which is faster than that of previous methods.This speed enabled us to achieve real-time seabed sediment recognition from SSS imagery.
基金supported by the National Key R&D Program of China(Grant No.2023YFC3010803)the National Nature Science Foundation of China(Grant No.52272424)+1 种基金the Key R&D Program of Hubei Province of China(Grant No.2023BCB123)the Fundamental Research Funds for the Central Universities(Grant No.WUT:2023IVB079)。
文摘Side-scan sonar(SSS)is now a prevalent instrument for large-scale seafloor topography measurements,deployable on an autonomous underwater vehicle(AUV)to execute fully automated underwater acoustic scanning imaging along a predetermined trajectory.However,SSS images often suffer from speckle noise caused by mutual interference between echoes,and limited AUV computational resources further hinder noise suppression.Existing approaches for SSS image processing and speckle noise reduction rely heavily on complex network structures and fail to combine the benefits of deep learning and domain knowledge.To address the problem,Rep DNet,a novel and effective despeckling convolutional neural network is proposed.Rep DNet introduces two re-parameterized blocks:the Pixel Smoothing Block(PSB)and Edge Enhancement Block(EEB),preserving edge information while attenuating speckle noise.During training,PSB and EEB manifest as double-layered multi-branch structures,integrating first-order and secondorder derivatives and smoothing functions.During inference,the branches are re-parameterized into a 3×3 convolution,enabling efficient inference without sacrificing accuracy.Rep DNet comprises three computational operations:3×3 convolution,element-wise summation and Rectified Linear Unit activation.Evaluations on benchmark datasets,a real SSS dataset and Data collected at Lake Mulan aestablish Rep DNet as a well-balanced network,meeting the AUV computational constraints in terms of performance and latency.
文摘Multi-beam Sonar and Side-scan Sonar compensate each other. In order to fully utilize all information, it is necessary to fuse two kinds of image and data. And the image co-registration is an important and complicated job before fusion. This paper suggests combining bathymetric data with intensity image, obtaining the characteristic points through the minimal angles of lines, and then deciding the corresponding image points by the maximal correlate coefficient in searching space. Finally, the second order polynomial is applied to the deformation model. After the images have been co-registered, Wavelet is used to fuse the images. It is shown that this algorithm can be used in the flat seafloor or the isotropic seabed. Verification is made in the paper with the observed data.
基金the National High Technology Project of China Foundation under Grant No.2002AA602230-1
文摘A multi-beam chirp sonar based on IP connections and DSP processing nodes was proposed and designed to provide an expandable system with high-speed processing and mass-storage of real-time signals for multi-beam profiling sonar.The system was designed for seabed petroleum pipeline detection and orientation,and can receive echo signals and process the data in real time,refreshing the display 10 times per second.Every node of the chirp sonar connects with data processing nodes through TCP/IP. Merely by adding nodes,the system’s processing ability can be increased proportionately without changing the software.System debugging and experimental testing proved the system to be practical and stable.This design provides a new method for high speed active sonar.
基金Financially supported by the NSFC (Grant No.40276022), KnowledgeInnovation Program of Chinese Academy of Sciences (KZCX3-SW-219)and JSPS international cooperation program, and the Ministry of Scienceand Technology Project (G200046704)
文摘Side-scan sonar data collected by Cruises 99-09 Leg 2 and 00-06 Leg l of R/V Yokosuka were used to reveal the sedimentary processes in Zenisu deep-sea channel. The middle and lower segments of the channel are rich in turbidite and other debrite deposits. By high-resolution imaging, three sedimentary processes were distinguished with distinct acoustic features. 1. Slumps and slides occur with contrasting backscatter, rough surface textures, blockings, and acoustic shadows at headwalls. They are very extensive and often in lobate form downslope. 2. Debris flow has uniform, general medium backscatter, sometimes showing marbling/lineation in lobate form. 3. Turbidity current is characterized by low backscatter confined to the channel as acoustic signal is attenuated. Regional tectonics must be the dominating factor that controls deposition pattern in this area.
