Presented in this study is a novel method for estimating the depth of single underwater source in shallow water,utilizing vector sensors.The approach leverages the depth distribution of the broadband Stokes parameters...Presented in this study is a novel method for estimating the depth of single underwater source in shallow water,utilizing vector sensors.The approach leverages the depth distribution of the broadband Stokes parameters to estimate source depth accurately.Unlike traditional matched field processing(MFP)and matched mode processing(MMP),the proposed approach can estimate source depth directly from the data received by sensors without requiring complete environmental information.Firstly,the broadband Stokes parameters(BSP)are established using the normal mode theory.Then the nonstationary phase approximation is used to simplify the theoretical derivation,which is necessary when dealing with broadband integrals.Additionally,range terms of the BSP are eliminated by normalization.By analyzing the depth distribution of the normalized broadband Stokes parameters(NBSP),it is found that the NBSP exhibit extreme values at the source depth,which can be used for source depth estimation.So the proposed depth estimation method is based on searching the peaks of the NBSP.Simulations show that this method is effective in relatively simple shallow water environments.Finally,the effect of source range,frequency bandwidth,sound speed profile(SSP),water depth,and signal-to-noise ratio(SNR)are studied.The findings indicate that the proposed method can accurately estimate the source depth when the SNR is greater than-5 d B and does not need to consider model mismatch issues.Additionally,variations in environmental parameters have minimal impact on estimation accuracy.Compared to MFP,the proposed method requires a higher SNR,but demonstrates superior robustness against fluctuations in environmental parameters.展开更多
Accurately estimating depth from underwater monocular images is essential for the target tracking task of unmanned underwater vehicles.This work proposes a method based on the Lpg-Lap Unet architecture.First,the Unet ...Accurately estimating depth from underwater monocular images is essential for the target tracking task of unmanned underwater vehicles.This work proposes a method based on the Lpg-Lap Unet architecture.First,the Unet architecture integrates Laplacian pyramid depth residuals and Sobel operators to improve the boundary details in depth images,which may suffer from the feature loss caused by upsampling and the blurriness of underwater images.Multiscale local planar guidance layers then fully exploit the intermediate depth features,and a comprehensive loss function ensures robustness and accuracy.Experimental results on benchmarks demonstrate the effectiveness of Lpg-Lap Unet and its superior performance over state-of-the-art models.An underwater target tracking system is then designed to further validate its real-time capabilities in the AirSim simulation platform.展开更多
Object detection in occluded environments remains a core challenge in computer vision(CV),especially in domains such as autonomous driving and robotics.While Convolutional Neural Network(CNN)-based twodimensional(2D)a...Object detection in occluded environments remains a core challenge in computer vision(CV),especially in domains such as autonomous driving and robotics.While Convolutional Neural Network(CNN)-based twodimensional(2D)and three-dimensional(3D)object detection methods havemade significant progress,they often fall short under severe occlusion due to depth ambiguities in 2D imagery and the high cost and deployment limitations of 3D sensors such as Light Detection and Ranging(LiDAR).This paper presents a comparative review of recent 2D and 3D detection models,focusing on their occlusion-handling capabilities and the impact of sensor modalities such as stereo vision,Time-of-Flight(ToF)cameras,and LiDAR.In this context,we introduce FuDensityNet,our multimodal occlusion-aware detection framework that combines Red-Green-Blue(RGB)images and LiDAR data to enhance detection performance.As a forward-looking direction,we propose a monocular depth-estimation extension to FuDensityNet,aimed at replacing expensive 3D sensors with a more scalable CNN-based pipeline.Although this enhancement is not experimentally evaluated in this manuscript,we describe its conceptual design and potential for future implementation.展开更多
Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain su...Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain suffer from inherent limitations:existing pose network branches infer camera ego-motion exclusively under static-scene and Lambertian-surface assumptions.These assumptions are often violated in real-world scenarios due to dynamic objects,non-Lambertian reflectance,and unstructured background elements,leading to pervasive artifacts such as depth discontinuities(“holes”),structural collapse,and ambiguous reconstruction.To address these challenges,we propose a novel framework that integrates scene dynamic pose estimation into the conventional self-supervised depth network,enhancing its ability to model complex scene dynamics.Our contributions are threefold:(1)a pixel-wise dynamic pose estimation module that jointly resolves the pose transformations of moving objects and localized scene perturbations;(2)a physically-informed loss function that couples dynamic pose and depth predictions,designed to mitigate depth errors arising from high-speed distant objects and geometrically inconsistent motion profiles;(3)an efficient SE(3)transformation parameterization that streamlines network complexity and temporal pre-processing.Extensive experiments on the KITTI and NYU-V2 benchmarks show that our framework achieves state-of-the-art performance in both quantitative metrics and qualitative visual fidelity,significantly improving the robustness and generalization of monocular depth estimation under dynamic conditions.展开更多
Precise and robust three-dimensional object detection(3DOD)presents a promising opportunity in the field of mobile robot(MR)navigation.Monocular 3DOD techniques typically involve extending existing twodimensional obje...Precise and robust three-dimensional object detection(3DOD)presents a promising opportunity in the field of mobile robot(MR)navigation.Monocular 3DOD techniques typically involve extending existing twodimensional object detection(2DOD)frameworks to predict the three-dimensional bounding box(3DBB)of objects captured in 2D RGB images.However,these methods often require multiple images,making them less feasible for various real-time scenarios.To address these challenges,the emergence of agile convolutional neural networks(CNNs)capable of inferring depth froma single image opens a new avenue for investigation.The paper proposes a novel ELDENet network designed to produce cost-effective 3DBounding Box Estimation(3D-BBE)froma single image.This novel framework comprises the PP-LCNet as the encoder and a fast convolutional decoder.Additionally,this integration includes a Squeeze-Exploit(SE)module utilizing the Math Kernel Library for Deep Neural Networks(MKLDNN)optimizer to enhance convolutional efficiency and streamline model size during effective training.Meanwhile,the proposed multi-scale sub-pixel decoder generates high-quality depth maps while maintaining a compact structure.Furthermore,the generated depthmaps provide a clear perspective with distance details of objects in the environment.These depth insights are combined with 2DOD for precise evaluation of 3D Bounding Boxes(3DBB),facilitating scene understanding and optimal route planning for mobile robots.Based on the estimated object center of the 3DBB,the Deep Reinforcement Learning(DRL)-based obstacle avoidance strategy for MRs is developed.Experimental results demonstrate that our model achieves state-of-the-art performance across three datasets:NYU-V2,KITTI,and Cityscapes.Overall,this framework shows significant potential for adaptation in intelligent mechatronic systems,particularly in developing knowledge-driven systems for mobile robot navigation.展开更多
Depth maps play a crucial role in various practical applications such as computer vision,augmented reality,and autonomous driving.How to obtain clear and accurate depth information in video depth estimation is a signi...Depth maps play a crucial role in various practical applications such as computer vision,augmented reality,and autonomous driving.How to obtain clear and accurate depth information in video depth estimation is a significant challenge faced in the field of computer vision.However,existing monocular video depth estimation models tend to produce blurred or inaccurate depth information in regions with object edges and low texture.To address this issue,we propose a monocular depth estimation model architecture guided by semantic segmentation masks,which introduces semantic information into the model to correct the ambiguous depth regions.We have evaluated the proposed method,and experimental results show that our method improves the accuracy of edge depth,demonstrating the effectiveness of our approach.展开更多
We present an estimation of depth of anomalous bodies using normalized full gradient （NFG） of gravity anomaly. Maxima in the NFG map can locate the bodies and indicate their depth. Model calculation using a sphere a...We present an estimation of depth of anomalous bodies using normalized full gradient （NFG） of gravity anomaly. Maxima in the NFG map can locate the bodies and indicate their depth. Model calculation using a sphere and application of the NFG method to gravity anomalies over salt domes in the USA and Denmark shows effectiveness of the method. However, the accuracy of depth estimation strongly depends on the number of term N in the Fourier series used to calculate the NFG. An optimum N for the calculation can be given from a test.展开更多
A method of source depth estimation based on the multi-path time delay difference is proposed. When the minimum time arrivals in all receiver depths are snapped to a certain time on time delay-depth plane, time delay ...A method of source depth estimation based on the multi-path time delay difference is proposed. When the minimum time arrivals in all receiver depths are snapped to a certain time on time delay-depth plane, time delay arrivals of surface-bottom reflection and bottom-surface reflection intersect at the source depth. Two hydrophones deployed vertically with a certain interval are required at least. If the receiver depths are known, the pair of time delays can be used to estimate the source depth. With the proposed method the source depth can be estimated successfully in a moderate range in the deep ocean without complicated matched-field calculations in the simulations and experiments.展开更多
Mineral exploration is done by different methods. Geophysical and geochemical studies are two powerful tools in this field. In integrated studies, the results of each study are used to determine the location of the dr...Mineral exploration is done by different methods. Geophysical and geochemical studies are two powerful tools in this field. In integrated studies, the results of each study are used to determine the location of the drilling boreholes. The purpose of this study is to use field geophysics to calculate the depth of mineral reserve. The study area is located 38 km from Zarand city called Jalalabad iron mine. In this study, gravimetric data were measured and mineral depth was calculated using the Euler method. 1314 readings have been performed in this area. The rocks of the region include volcanic and sedimentary. The source of the mineralization in the area is hydrothermal processes. After gravity measuring in the region, the data were corrected, then various methods such as anomalous map remaining in levels one and two, upward expansion, first and second-degree vertical derivatives, analytical method, and analytical signal were drawn, and finally, the depth of the deposit was estimated by Euler method. As a result, the depth of the mineral deposit was calculated to be between 20 and 30 meters on average.展开更多
In this paper, we propose a new algorithm for temporally consistent depth map estimation to generate three-dimensional video. The proposed algorithm adaptively computes the matching cost using a temporal weighting fun...In this paper, we propose a new algorithm for temporally consistent depth map estimation to generate three-dimensional video. The proposed algorithm adaptively computes the matching cost using a temporal weighting function, which is obtained by block-based moving object detection and motion estimation with variable block sizes. Experimental results show that the proposed algorithm improves the temporal consistency of the depth video and reduces by about 38% both the flickering artefact in the synthesized view and the number of coding bits for depth video coding.展开更多
Depth estimation of subsurface faults is one of the problems in gravity interpretation. We tried using the support vector classifier （SVC） method in the estimation. Using forward and nonlinear inverse techniques, de...Depth estimation of subsurface faults is one of the problems in gravity interpretation. We tried using the support vector classifier （SVC） method in the estimation. Using forward and nonlinear inverse techniques, detecting the depth of subsurface faults with related error is possible but it is necessary to have an initial guess for the depth and this initial guess usually comes from non-gravity data. We introduce SVC in this paper as one of the tools for estimating the depth of subsurface faults using gravity data. We can suppose that each subsurface fault depth is a class and that SVC is a classification algorithm. To better use the SVC algorithm, we select proper depth estimation features using a proper features selection （FS） algorithm. In this research, we produce a training set consisting of synthetic gravity profiles created by subsurface faults at different depths to train the SVC code to estimate the depth of real subsurface faults. Then we test our trained SVC code by a testing set consisting of other synthetic gravity profiles created by subsurface faults at different depths. We also tested our trained SVC code using real data.展开更多
Recent advances in computer vision and deep learning have shown that the fusion of depth information can significantly enhance the performance of RGB-based damage detection and segmentation models.However,alongside th...Recent advances in computer vision and deep learning have shown that the fusion of depth information can significantly enhance the performance of RGB-based damage detection and segmentation models.However,alongside the advantages,depth-sensing also presents many practical challenges.For instance,the depth sensors impose an additional payload burden on the robotic inspection platforms limiting the operation time and increasing the inspection cost.Additionally,some lidar-based depth sensors have poor outdoor performance due to sunlight contamination during the daytime.In this context,this study investigates the feasibility of abolishing depth-sensing at test time without compromising the segmentation performance.An autonomous damage segmentation framework is developed,based on recent advancements in vision-based multi-modal sensing such as modality hallucination(MH)and monocular depth estimation(MDE),which require depth data only during the model training.At the time of deployment,depth data becomes expendable as it can be simulated from the corresponding RGB frames.This makes it possible to reap the benefits of depth fusion without any depth perception per se.This study explored two different depth encoding techniques and three different fusion strategies in addition to a baseline RGB-based model.The proposed approach is validated on computer-generated RGB-D data of reinforced concrete buildings subjected to seismic damage.It was observed that the surrogate techniques can increase the segmentation IoU by up to 20.1%with a negligible increase in the computation cost.Overall,this study is believed to make a positive contribution to enhancing the resilience of critical civil infrastructure.展开更多
Monocular depth estimation is the basic task in computer vision.Its accuracy has tremendous improvement in the decade with the development of deep learning.However,the blurry boundary in the depth map is a serious pro...Monocular depth estimation is the basic task in computer vision.Its accuracy has tremendous improvement in the decade with the development of deep learning.However,the blurry boundary in the depth map is a serious problem.Researchers find that the blurry boundary is mainly caused by two factors.First,the low-level features,containing boundary and structure information,may be lost in deep networks during the convolution process.Second,themodel ignores the errors introduced by the boundary area due to the few portions of the boundary area in the whole area,during the backpropagation.Focusing on the factors mentioned above.Two countermeasures are proposed to mitigate the boundary blur problem.Firstly,we design a scene understanding module and scale transformmodule to build a lightweight fuse feature pyramid,which can deal with low-level feature loss effectively.Secondly,we propose a boundary-aware depth loss function to pay attention to the effects of the boundary’s depth value.Extensive experiments show that our method can predict the depth maps with clearer boundaries,and the performance of the depth accuracy based on NYU-Depth V2,SUN RGB-D,and iBims-1 are competitive.展开更多
Depth estimation is an active research area with the developing of stereo vision in recent years. It is one of the key technologies to resolve the large data of stereo vision communication. Now depth estimation still ...Depth estimation is an active research area with the developing of stereo vision in recent years. It is one of the key technologies to resolve the large data of stereo vision communication. Now depth estimation still has some problems, such as occlusion, fuzzy edge, real-time processing, etc. Many algorithms have been proposed base on software, however the performance of the computer configurations limits the software processing speed. The other resolution is hardware design and the great developments of the digital signal processor （DSP）, and application specific integrated circuit （ASIC） and field programmable gate array （FPGA） provide the opportunity of flexible applications. In this work, by analyzing the procedures of depth estimation, the proper algorithms which can be used in hardware design to execute real-time depth estimation are proposed. The different methods of calibration, matching and post-processing are analyzed based on the hardware design requirements. At last some tests for the algorithm have been analyzed. The results show that the algorithms proposed for hardware design can provide credited depth map for further view synthesis and are suitable for hardware design.展开更多
Learning-based multi-task models have been widely used in various scene understanding tasks,and complement each other,i.e.,they allow us to consider prior semantic information to better infer depth.We boost the unsupe...Learning-based multi-task models have been widely used in various scene understanding tasks,and complement each other,i.e.,they allow us to consider prior semantic information to better infer depth.We boost the unsupervised monocular depth estimation using semantic segmentation as an auxiliary task.To address the lack of cross-domain datasets and catastrophic forgetting problems encountered in multi-task training,we utilize existing methodology to obtain redundant segmentation maps to build our cross-domain dataset,which not only provides a new way to conduct multi-task training,but also helps us to evaluate results compared with those of other algorithms.In addition,in order to comprehensively use the extracted features of the two tasks in the early perception stage,we use a strategy of sharing weights in the network to fuse cross-domain features,and introduce a novel multi-task loss function to further smooth the depth values.Extensive experiments on KITTI and Cityscapes datasets show that our method has achieved state-of-the-art performance in the depth estimation task,as well improved semantic segmentation.展开更多
Depth estimation is an important task in computer vision.Collecting data at scale for monocular depth estimation is challenging,as this task requires simultaneously capturing RGB images and depth information.Therefore...Depth estimation is an important task in computer vision.Collecting data at scale for monocular depth estimation is challenging,as this task requires simultaneously capturing RGB images and depth information.Therefore,data augmentation is crucial for this task.Existing data augmentationmethods often employ pixel-wise transformations,whichmay inadvertently disrupt edge features.In this paper,we propose a data augmentationmethod formonocular depth estimation,which we refer to as the Perpendicular-Cutdepth method.This method involves cutting realworld depth maps along perpendicular directions and pasting them onto input images,thereby diversifying the data without compromising edge features.To validate the effectiveness of the algorithm,we compared it with existing convolutional neural network(CNN)against the current mainstream data augmentation algorithms.Additionally,to verify the algorithm’s applicability to Transformer networks,we designed an encoder-decoder network structure based on Transformer to assess the generalization of our proposed algorithm.Experimental results demonstrate that,in the field of monocular depth estimation,our proposed method,Perpendicular-Cutdepth,outperforms traditional data augmentationmethods.On the indoor dataset NYU,our method increases accuracy from0.900 to 0.907 and reduces the error rate from0.357 to 0.351.On the outdoor dataset KITTI,our method improves accuracy from 0.9638 to 0.9642 and decreases the error rate from 0.060 to 0.0598.展开更多
For traffic object detection in foggy environment based on convolutional neural network(CNN),data sets in fog-free environment are generally used to train the network directly.As a result,the network cannot learn the ...For traffic object detection in foggy environment based on convolutional neural network(CNN),data sets in fog-free environment are generally used to train the network directly.As a result,the network cannot learn the object characteristics in the foggy environment in the training set,and the detection effect is not good.To improve the traffic object detection in foggy environment,we propose a method of generating foggy images on fog-free images from the perspective of data set construction.First,taking the KITTI objection detection data set as an original fog-free image,we generate the depth image of the original image by using improved Monodepth unsupervised depth estimation method.Then,a geometric prior depth template is constructed to fuse the image entropy taken as weight with the depth image.After that,a foggy image is acquired from the depth image based on the atmospheric scattering model.Finally,we take two typical object-detection frameworks,that is,the two-stage object-detection Fster region-based convolutional neural network(Faster-RCNN)and the one-stage object-detection network YOLOv4,to train the original data set,the foggy data set and the mixed data set,respectively.According to the test results on RESIDE-RTTS data set in the outdoor natural foggy environment,the model under the training on the mixed data set shows the best effect.The mean average precision(mAP)values are increased by 5.6%and by 5.0%under the YOLOv4 model and the Faster-RCNN network,respectively.It is proved that the proposed method can effectively improve object identification ability foggy environment.展开更多
Most approaches to estimate a scene’s 3D depth from a single image often model the point spread function (PSF) as a 2D Gaussian function. However, those method<span>s</span><span> are suffered ...Most approaches to estimate a scene’s 3D depth from a single image often model the point spread function (PSF) as a 2D Gaussian function. However, those method<span>s</span><span> are suffered from some noises, and difficult to get a high quality of depth recovery. We presented a simple yet effective approach to estimate exactly the amount of spatially varying defocus blur at edges, based on </span><span>a</span><span> Cauchy distribution model for the PSF. The raw image was re-blurred twice using two known Cauchy distribution kernels, and the defocus blur amount at edges could be derived from the gradient ratio between the two re-blurred images. By propagating the blur amount at edge locations to the entire image using the matting interpolation, a full depth map was then recovered. Experimental results on several real images demonstrated both feasibility and effectiveness of our method, being a non-Gaussian model for DSF, in providing a better estimation of the defocus map from a single un-calibrated defocused image. These results also showed that our method </span><span>was</span><span> robust to image noises, inaccurate edge location and interferences of neighboring edges. It could generate more accurate scene depth maps than the most of existing methods using a Gaussian based DSF model.</span>展开更多
Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods dire...Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.展开更多
With rapid development of UAV technology,research on UAV image analysis has gained attention.As the existing techniques of UAV target localization often rely on additional equipment,a method of UAV target localization...With rapid development of UAV technology,research on UAV image analysis has gained attention.As the existing techniques of UAV target localization often rely on additional equipment,a method of UAV target localization based on depth estimation has been proposed.However,the unique perspective of UAVs poses challenges such as the significant field of view variations and the presence of dynamic objects in the scene.As a result,the existing methods of depth estimation and scale recovery cannot be directly applied to UAV perspectives.Additionally,there is a scarcity of depth estimation datasets tailored for UAV perspectives,which makes supervised algorithms impractical.To address these issues,an outlier filter is introduced to enhance the applicability of depth estimation networks to target localization.A frame buffer method is proposed to achieve more accurate scale recovery,so as to handle complex scene textures in UAV images.The proposed method demonstrates a 14.29%improvement over the baseline.Compared with the average recovery results from UAV perspectives,the difference is only 0.88%,approaching the performance of scale recovery using ground truth labels.Furthermore,to overcome the limited availability of traditional UAV depth datasets,a method for generating depth labels from video sequences is proposed.Compared to state-of-the-art methods,the proposed approach achieves higher accuracy in depth estimation and stands for the first attempt at target localization using image sequences.Proposed algorithm and dataset are available at https://github.com/uav-tan/uav-object-localization.展开更多
基金Project supported by the National Natural Science Foundation of China(Grant Nos.12274348 and 12004335)the National Key Research and Development Program of China(Grant No.2024YFC2813800)。
文摘Presented in this study is a novel method for estimating the depth of single underwater source in shallow water,utilizing vector sensors.The approach leverages the depth distribution of the broadband Stokes parameters to estimate source depth accurately.Unlike traditional matched field processing(MFP)and matched mode processing(MMP),the proposed approach can estimate source depth directly from the data received by sensors without requiring complete environmental information.Firstly,the broadband Stokes parameters(BSP)are established using the normal mode theory.Then the nonstationary phase approximation is used to simplify the theoretical derivation,which is necessary when dealing with broadband integrals.Additionally,range terms of the BSP are eliminated by normalization.By analyzing the depth distribution of the normalized broadband Stokes parameters(NBSP),it is found that the NBSP exhibit extreme values at the source depth,which can be used for source depth estimation.So the proposed depth estimation method is based on searching the peaks of the NBSP.Simulations show that this method is effective in relatively simple shallow water environments.Finally,the effect of source range,frequency bandwidth,sound speed profile(SSP),water depth,and signal-to-noise ratio(SNR)are studied.The findings indicate that the proposed method can accurately estimate the source depth when the SNR is greater than-5 d B and does not need to consider model mismatch issues.Additionally,variations in environmental parameters have minimal impact on estimation accuracy.Compared to MFP,the proposed method requires a higher SNR,but demonstrates superior robustness against fluctuations in environmental parameters.
基金partially supported by the Natural Science Foundation of Shandong Province,China(No.ZR2023ME009)the National Natural Science Foundation of China(No.51909252)。
文摘Accurately estimating depth from underwater monocular images is essential for the target tracking task of unmanned underwater vehicles.This work proposes a method based on the Lpg-Lap Unet architecture.First,the Unet architecture integrates Laplacian pyramid depth residuals and Sobel operators to improve the boundary details in depth images,which may suffer from the feature loss caused by upsampling and the blurriness of underwater images.Multiscale local planar guidance layers then fully exploit the intermediate depth features,and a comprehensive loss function ensures robustness and accuracy.Experimental results on benchmarks demonstrate the effectiveness of Lpg-Lap Unet and its superior performance over state-of-the-art models.An underwater target tracking system is then designed to further validate its real-time capabilities in the AirSim simulation platform.
文摘Object detection in occluded environments remains a core challenge in computer vision(CV),especially in domains such as autonomous driving and robotics.While Convolutional Neural Network(CNN)-based twodimensional(2D)and three-dimensional(3D)object detection methods havemade significant progress,they often fall short under severe occlusion due to depth ambiguities in 2D imagery and the high cost and deployment limitations of 3D sensors such as Light Detection and Ranging(LiDAR).This paper presents a comparative review of recent 2D and 3D detection models,focusing on their occlusion-handling capabilities and the impact of sensor modalities such as stereo vision,Time-of-Flight(ToF)cameras,and LiDAR.In this context,we introduce FuDensityNet,our multimodal occlusion-aware detection framework that combines Red-Green-Blue(RGB)images and LiDAR data to enhance detection performance.As a forward-looking direction,we propose a monocular depth-estimation extension to FuDensityNet,aimed at replacing expensive 3D sensors with a more scalable CNN-based pipeline.Although this enhancement is not experimentally evaluated in this manuscript,we describe its conceptual design and potential for future implementation.
基金supported in part by the National Natural Science Foundation of China under Grants 62071345。
文摘Self-supervised monocular depth estimation has emerged as a major research focus in recent years,primarily due to the elimination of ground-truth depth dependence.However,the prevailing architectures in this domain suffer from inherent limitations:existing pose network branches infer camera ego-motion exclusively under static-scene and Lambertian-surface assumptions.These assumptions are often violated in real-world scenarios due to dynamic objects,non-Lambertian reflectance,and unstructured background elements,leading to pervasive artifacts such as depth discontinuities(“holes”),structural collapse,and ambiguous reconstruction.To address these challenges,we propose a novel framework that integrates scene dynamic pose estimation into the conventional self-supervised depth network,enhancing its ability to model complex scene dynamics.Our contributions are threefold:(1)a pixel-wise dynamic pose estimation module that jointly resolves the pose transformations of moving objects and localized scene perturbations;(2)a physically-informed loss function that couples dynamic pose and depth predictions,designed to mitigate depth errors arising from high-speed distant objects and geometrically inconsistent motion profiles;(3)an efficient SE(3)transformation parameterization that streamlines network complexity and temporal pre-processing.Extensive experiments on the KITTI and NYU-V2 benchmarks show that our framework achieves state-of-the-art performance in both quantitative metrics and qualitative visual fidelity,significantly improving the robustness and generalization of monocular depth estimation under dynamic conditions.
文摘Precise and robust three-dimensional object detection(3DOD)presents a promising opportunity in the field of mobile robot(MR)navigation.Monocular 3DOD techniques typically involve extending existing twodimensional object detection(2DOD)frameworks to predict the three-dimensional bounding box(3DBB)of objects captured in 2D RGB images.However,these methods often require multiple images,making them less feasible for various real-time scenarios.To address these challenges,the emergence of agile convolutional neural networks(CNNs)capable of inferring depth froma single image opens a new avenue for investigation.The paper proposes a novel ELDENet network designed to produce cost-effective 3DBounding Box Estimation(3D-BBE)froma single image.This novel framework comprises the PP-LCNet as the encoder and a fast convolutional decoder.Additionally,this integration includes a Squeeze-Exploit(SE)module utilizing the Math Kernel Library for Deep Neural Networks(MKLDNN)optimizer to enhance convolutional efficiency and streamline model size during effective training.Meanwhile,the proposed multi-scale sub-pixel decoder generates high-quality depth maps while maintaining a compact structure.Furthermore,the generated depthmaps provide a clear perspective with distance details of objects in the environment.These depth insights are combined with 2DOD for precise evaluation of 3D Bounding Boxes(3DBB),facilitating scene understanding and optimal route planning for mobile robots.Based on the estimated object center of the 3DBB,the Deep Reinforcement Learning(DRL)-based obstacle avoidance strategy for MRs is developed.Experimental results demonstrate that our model achieves state-of-the-art performance across three datasets:NYU-V2,KITTI,and Cityscapes.Overall,this framework shows significant potential for adaptation in intelligent mechatronic systems,particularly in developing knowledge-driven systems for mobile robot navigation.
文摘Depth maps play a crucial role in various practical applications such as computer vision,augmented reality,and autonomous driving.How to obtain clear and accurate depth information in video depth estimation is a significant challenge faced in the field of computer vision.However,existing monocular video depth estimation models tend to produce blurred or inaccurate depth information in regions with object edges and low texture.To address this issue,we propose a monocular depth estimation model architecture guided by semantic segmentation masks,which introduces semantic information into the model to correct the ambiguous depth regions.We have evaluated the proposed method,and experimental results show that our method improves the accuracy of edge depth,demonstrating the effectiveness of our approach.
基金supported by the Ministry of Science,Researches and Technology,Iran
文摘We present an estimation of depth of anomalous bodies using normalized full gradient （NFG） of gravity anomaly. Maxima in the NFG map can locate the bodies and indicate their depth. Model calculation using a sphere and application of the NFG method to gravity anomalies over salt domes in the USA and Denmark shows effectiveness of the method. However, the accuracy of depth estimation strongly depends on the number of term N in the Fourier series used to calculate the NFG. An optimum N for the calculation can be given from a test.
基金Supported by the National Natural Science Foundation of China under Grant No 11174235
文摘A method of source depth estimation based on the multi-path time delay difference is proposed. When the minimum time arrivals in all receiver depths are snapped to a certain time on time delay-depth plane, time delay arrivals of surface-bottom reflection and bottom-surface reflection intersect at the source depth. Two hydrophones deployed vertically with a certain interval are required at least. If the receiver depths are known, the pair of time delays can be used to estimate the source depth. With the proposed method the source depth can be estimated successfully in a moderate range in the deep ocean without complicated matched-field calculations in the simulations and experiments.
文摘Mineral exploration is done by different methods. Geophysical and geochemical studies are two powerful tools in this field. In integrated studies, the results of each study are used to determine the location of the drilling boreholes. The purpose of this study is to use field geophysics to calculate the depth of mineral reserve. The study area is located 38 km from Zarand city called Jalalabad iron mine. In this study, gravimetric data were measured and mineral depth was calculated using the Euler method. 1314 readings have been performed in this area. The rocks of the region include volcanic and sedimentary. The source of the mineralization in the area is hydrothermal processes. After gravity measuring in the region, the data were corrected, then various methods such as anomalous map remaining in levels one and two, upward expansion, first and second-degree vertical derivatives, analytical method, and analytical signal were drawn, and finally, the depth of the deposit was estimated by Euler method. As a result, the depth of the mineral deposit was calculated to be between 20 and 30 meters on average.
基金supported by the National Research Foundation of Korea Grant funded by the Korea Ministry of Science and Technology under Grant No. 2012-0009228
文摘In this paper, we propose a new algorithm for temporally consistent depth map estimation to generate three-dimensional video. The proposed algorithm adaptively computes the matching cost using a temporal weighting function, which is obtained by block-based moving object detection and motion estimation with variable block sizes. Experimental results show that the proposed algorithm improves the temporal consistency of the depth video and reduces by about 38% both the flickering artefact in the synthesized view and the number of coding bits for depth video coding.
文摘Depth estimation of subsurface faults is one of the problems in gravity interpretation. We tried using the support vector classifier （SVC） method in the estimation. Using forward and nonlinear inverse techniques, detecting the depth of subsurface faults with related error is possible but it is necessary to have an initial guess for the depth and this initial guess usually comes from non-gravity data. We introduce SVC in this paper as one of the tools for estimating the depth of subsurface faults using gravity data. We can suppose that each subsurface fault depth is a class and that SVC is a classification algorithm. To better use the SVC algorithm, we select proper depth estimation features using a proper features selection （FS） algorithm. In this research, we produce a training set consisting of synthetic gravity profiles created by subsurface faults at different depths to train the SVC code to estimate the depth of real subsurface faults. Then we test our trained SVC code by a testing set consisting of other synthetic gravity profiles created by subsurface faults at different depths. We also tested our trained SVC code using real data.
基金supported in part by a fund from Bentley Systems,Inc.
文摘Recent advances in computer vision and deep learning have shown that the fusion of depth information can significantly enhance the performance of RGB-based damage detection and segmentation models.However,alongside the advantages,depth-sensing also presents many practical challenges.For instance,the depth sensors impose an additional payload burden on the robotic inspection platforms limiting the operation time and increasing the inspection cost.Additionally,some lidar-based depth sensors have poor outdoor performance due to sunlight contamination during the daytime.In this context,this study investigates the feasibility of abolishing depth-sensing at test time without compromising the segmentation performance.An autonomous damage segmentation framework is developed,based on recent advancements in vision-based multi-modal sensing such as modality hallucination(MH)and monocular depth estimation(MDE),which require depth data only during the model training.At the time of deployment,depth data becomes expendable as it can be simulated from the corresponding RGB frames.This makes it possible to reap the benefits of depth fusion without any depth perception per se.This study explored two different depth encoding techniques and three different fusion strategies in addition to a baseline RGB-based model.The proposed approach is validated on computer-generated RGB-D data of reinforced concrete buildings subjected to seismic damage.It was observed that the surrogate techniques can increase the segmentation IoU by up to 20.1%with a negligible increase in the computation cost.Overall,this study is believed to make a positive contribution to enhancing the resilience of critical civil infrastructure.
基金supported in part by School Research Projects of Wuyi University (No.5041700175).
文摘Monocular depth estimation is the basic task in computer vision.Its accuracy has tremendous improvement in the decade with the development of deep learning.However,the blurry boundary in the depth map is a serious problem.Researchers find that the blurry boundary is mainly caused by two factors.First,the low-level features,containing boundary and structure information,may be lost in deep networks during the convolution process.Second,themodel ignores the errors introduced by the boundary area due to the few portions of the boundary area in the whole area,during the backpropagation.Focusing on the factors mentioned above.Two countermeasures are proposed to mitigate the boundary blur problem.Firstly,we design a scene understanding module and scale transformmodule to build a lightweight fuse feature pyramid,which can deal with low-level feature loss effectively.Secondly,we propose a boundary-aware depth loss function to pay attention to the effects of the boundary’s depth value.Extensive experiments show that our method can predict the depth maps with clearer boundaries,and the performance of the depth accuracy based on NYU-Depth V2,SUN RGB-D,and iBims-1 are competitive.
基金supported by the National Natural Science Foundation of China(Grant Nos.60832003)the Key Laboratory of Advanced Display and System Applications(Shanghai University),Ministry of Education,China(Grant No.P200801)the Science and Technology Commission of Shanghai Municipality(Grant No.10510500500)
文摘Depth estimation is an active research area with the developing of stereo vision in recent years. It is one of the key technologies to resolve the large data of stereo vision communication. Now depth estimation still has some problems, such as occlusion, fuzzy edge, real-time processing, etc. Many algorithms have been proposed base on software, however the performance of the computer configurations limits the software processing speed. The other resolution is hardware design and the great developments of the digital signal processor （DSP）, and application specific integrated circuit （ASIC） and field programmable gate array （FPGA） provide the opportunity of flexible applications. In this work, by analyzing the procedures of depth estimation, the proper algorithms which can be used in hardware design to execute real-time depth estimation are proposed. The different methods of calibration, matching and post-processing are analyzed based on the hardware design requirements. At last some tests for the algorithm have been analyzed. The results show that the algorithms proposed for hardware design can provide credited depth map for further view synthesis and are suitable for hardware design.
基金This work was supported by the national key research development plan(Project No.YS2018YFB1403703)research project of the communication university of china(Project No.CUC200D058).
文摘Learning-based multi-task models have been widely used in various scene understanding tasks,and complement each other,i.e.,they allow us to consider prior semantic information to better infer depth.We boost the unsupervised monocular depth estimation using semantic segmentation as an auxiliary task.To address the lack of cross-domain datasets and catastrophic forgetting problems encountered in multi-task training,we utilize existing methodology to obtain redundant segmentation maps to build our cross-domain dataset,which not only provides a new way to conduct multi-task training,but also helps us to evaluate results compared with those of other algorithms.In addition,in order to comprehensively use the extracted features of the two tasks in the early perception stage,we use a strategy of sharing weights in the network to fuse cross-domain features,and introduce a novel multi-task loss function to further smooth the depth values.Extensive experiments on KITTI and Cityscapes datasets show that our method has achieved state-of-the-art performance in the depth estimation task,as well improved semantic segmentation.
基金the Grant of Program for Scientific ResearchInnovation Team in Colleges and Universities of Anhui Province(2022AH010095)The Grant ofScientific Research and Talent Development Foundation of the Hefei University(No.21-22RC15)+2 种基金The Key Research Plan of Anhui Province(No.2022k07020011)The Grant of Anhui Provincial940 CMC,2024,vol.79,no.1Natural Science Foundation,No.2308085MF213The Open Fund of Information Materials andIntelligent Sensing Laboratory of Anhui Province IMIS202205,as well as the AI General ComputingPlatform of Hefei University.
文摘Depth estimation is an important task in computer vision.Collecting data at scale for monocular depth estimation is challenging,as this task requires simultaneously capturing RGB images and depth information.Therefore,data augmentation is crucial for this task.Existing data augmentationmethods often employ pixel-wise transformations,whichmay inadvertently disrupt edge features.In this paper,we propose a data augmentationmethod formonocular depth estimation,which we refer to as the Perpendicular-Cutdepth method.This method involves cutting realworld depth maps along perpendicular directions and pasting them onto input images,thereby diversifying the data without compromising edge features.To validate the effectiveness of the algorithm,we compared it with existing convolutional neural network(CNN)against the current mainstream data augmentation algorithms.Additionally,to verify the algorithm’s applicability to Transformer networks,we designed an encoder-decoder network structure based on Transformer to assess the generalization of our proposed algorithm.Experimental results demonstrate that,in the field of monocular depth estimation,our proposed method,Perpendicular-Cutdepth,outperforms traditional data augmentationmethods.On the indoor dataset NYU,our method increases accuracy from0.900 to 0.907 and reduces the error rate from0.357 to 0.351.On the outdoor dataset KITTI,our method improves accuracy from 0.9638 to 0.9642 and decreases the error rate from 0.060 to 0.0598.
文摘For traffic object detection in foggy environment based on convolutional neural network(CNN),data sets in fog-free environment are generally used to train the network directly.As a result,the network cannot learn the object characteristics in the foggy environment in the training set,and the detection effect is not good.To improve the traffic object detection in foggy environment,we propose a method of generating foggy images on fog-free images from the perspective of data set construction.First,taking the KITTI objection detection data set as an original fog-free image,we generate the depth image of the original image by using improved Monodepth unsupervised depth estimation method.Then,a geometric prior depth template is constructed to fuse the image entropy taken as weight with the depth image.After that,a foggy image is acquired from the depth image based on the atmospheric scattering model.Finally,we take two typical object-detection frameworks,that is,the two-stage object-detection Fster region-based convolutional neural network(Faster-RCNN)and the one-stage object-detection network YOLOv4,to train the original data set,the foggy data set and the mixed data set,respectively.According to the test results on RESIDE-RTTS data set in the outdoor natural foggy environment,the model under the training on the mixed data set shows the best effect.The mean average precision(mAP)values are increased by 5.6%and by 5.0%under the YOLOv4 model and the Faster-RCNN network,respectively.It is proved that the proposed method can effectively improve object identification ability foggy environment.
文摘Most approaches to estimate a scene’s 3D depth from a single image often model the point spread function (PSF) as a 2D Gaussian function. However, those method<span>s</span><span> are suffered from some noises, and difficult to get a high quality of depth recovery. We presented a simple yet effective approach to estimate exactly the amount of spatially varying defocus blur at edges, based on </span><span>a</span><span> Cauchy distribution model for the PSF. The raw image was re-blurred twice using two known Cauchy distribution kernels, and the defocus blur amount at edges could be derived from the gradient ratio between the two re-blurred images. By propagating the blur amount at edge locations to the entire image using the matting interpolation, a full depth map was then recovered. Experimental results on several real images demonstrated both feasibility and effectiveness of our method, being a non-Gaussian model for DSF, in providing a better estimation of the defocus map from a single un-calibrated defocused image. These results also showed that our method </span><span>was</span><span> robust to image noises, inaccurate edge location and interferences of neighboring edges. It could generate more accurate scene depth maps than the most of existing methods using a Gaussian based DSF model.</span>
基金Supported by the National Natural Science Foundation of China under Grants 61872241, 62077037 and 62077037Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0102。
文摘Background Monocular depth estimation aims to predict a dense depth map from a single RGB image,and has important applications in 3D reconstruction,automatic driving,and augmented reality.However,existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy,which leads to inferior performance.Methods To remove the influence of depth-irrelevant information and improve the depth-prediction accuracy,we propose RADepthNet,a novel reflectance-guided network that fuses boundary features.Specifically,our method predicts depth maps using the following three steps:(1)Intrinsic Image Decomposition.We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance.Through an ablation study,we demonstrate that the module can reduce the influence of illumination on depth estimation.(2)Boundary Detection.A boundary extraction module,consisting of an encoder,refinement block,and upsample block,was proposed to better predict the depth at object boundaries utilizing gradient constraints.(3)Depth Prediction Module.We use an encoder different from(2)to obtain depth features from the reflectance map and fuse boundary features to predict depth.In addition,we proposed FIFADataset,a depth-estimation dataset applied in soccer scenarios.Results Extensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.
基金financial support from the National Key Research and Development Program of China (Nos. 2022YFB3904303 and 2020YFB0505602)the National Natural Science Foundation of China (Nos. 62076019, 62022012, U2233217, 62101019 and 62371029)the Civil Aviation Security Capacity Building Fund Project, China (Nos. CAAC Contract 2020(123), CAAC Contract 2021(77) and CAAC Contract 2022(110))
文摘With rapid development of UAV technology,research on UAV image analysis has gained attention.As the existing techniques of UAV target localization often rely on additional equipment,a method of UAV target localization based on depth estimation has been proposed.However,the unique perspective of UAVs poses challenges such as the significant field of view variations and the presence of dynamic objects in the scene.As a result,the existing methods of depth estimation and scale recovery cannot be directly applied to UAV perspectives.Additionally,there is a scarcity of depth estimation datasets tailored for UAV perspectives,which makes supervised algorithms impractical.To address these issues,an outlier filter is introduced to enhance the applicability of depth estimation networks to target localization.A frame buffer method is proposed to achieve more accurate scale recovery,so as to handle complex scene textures in UAV images.The proposed method demonstrates a 14.29%improvement over the baseline.Compared with the average recovery results from UAV perspectives,the difference is only 0.88%,approaching the performance of scale recovery using ground truth labels.Furthermore,to overcome the limited availability of traditional UAV depth datasets,a method for generating depth labels from video sequences is proposed.Compared to state-of-the-art methods,the proposed approach achieves higher accuracy in depth estimation and stands for the first attempt at target localization using image sequences.Proposed algorithm and dataset are available at https://github.com/uav-tan/uav-object-localization.
