Image super-resolution reconstruction technology is currently widely used in medical imaging,video surveillance,and industrial quality inspection.It not only enhances image quality but also improves details and visual...Image super-resolution reconstruction technology is currently widely used in medical imaging,video surveillance,and industrial quality inspection.It not only enhances image quality but also improves details and visual perception,significantly increasing the utility of low-resolution images.In this study,an improved image superresolution reconstruction model based on Generative Adversarial Networks(SRGAN)was proposed.This model introduced a channel and spatial attention mechanism(CSAB)in the generator,allowing it to effectively leverage the information from the input image to enhance feature representations and capture important details.The discriminator was designed with an improved PatchGAN architecture,which more accurately captured local details and texture information of the image.With these enhanced generator and discriminator architectures and an optimized loss function design,this method demonstrated superior performance in image quality assessment metrics.Experimental results showed that this model outperforms traditional methods,presenting more detailed and realistic image details in the visual effects.展开更多
As a form of discrete representation learning,Vector Quantized Variational Autoencoders(VQ-VAE)have increasingly been applied to generative and multimodal tasks due to their ease of embedding and representative capaci...As a form of discrete representation learning,Vector Quantized Variational Autoencoders(VQ-VAE)have increasingly been applied to generative and multimodal tasks due to their ease of embedding and representative capacity.However,existing VQ-VAEs often perform quantization in the spatial domain,ignoring global structural information and potentially suffering from codebook collapse and information coupling issues.This paper proposes a frequency quantized variational autoencoder(FQ-VAE)to address these issues.The proposed method transforms image features into linear combinations in the frequency domain using a 2D fast Fourier transform(2D-FFT)and performs adaptive quantization on these frequency components to preserve image’s global relationships.The codebook is dynamically optimized to avoid collapse and information coupling issue by considering the usage frequency and dependency of code vectors.Furthermore,we introduce a post-processing module based on graph convolutional networks to further improve reconstruction quality.Experimental results on four public datasets demonstrate that the proposed method outperforms state-of-the-art approaches in terms of Structural Similarity Index(SSIM),Learned Perceptual Image Patch Similarity(LPIPS),and Reconstruction Fréchet Inception Distance(rFID).In the experiments on the CIFAR-10 dataset,compared to the baselinemethod VQ-VAE,the proposedmethod improves the abovemetrics by 4.9%,36.4%,and 52.8%,respectively.展开更多
Among hyperspectral imaging technologies, interferometric spectral imaging is widely used in remote sening due to advantages of large luminous flux and high resolution. However, with complicated mechanism, interferome...Among hyperspectral imaging technologies, interferometric spectral imaging is widely used in remote sening due to advantages of large luminous flux and high resolution. However, with complicated mechanism, interferometric imaging faces the impact of multi-stage degradation. Most exsiting interferometric spectrum reconstruction methods are based on tradition model-based framework with multiple steps, showing poor efficiency and restricted performance. Thus, we propose an interferometric spectrum reconstruction method based on degradation synthesis and deep learning.Firstly, based on imaging mechanism, we proposed an mathematical model of interferometric imaging to analyse the degradation components as noises and trends during imaging. The model consists of three stages, namely instrument degradation, sensing degradation, and signal-independent degradation process. Then, we designed calibration-based method to estimate parameters in the model, of which the results are used for synthesizing realistic dataset for learning-based algorithms.In addition, we proposed a dual-stage interferogram spectrum reconstruction framework, which supports pre-training and integration of denoising DNNs. Experiments exhibits the reliability of our degradation model and synthesized data, and the effectiveness of the proposed reconstruction method.展开更多
Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic ...Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic imaging techniques,such as bioluminescence tomography(BLT).Nevertheless,nearly every existing DL-based method utilizes an explicit neural representation for the reconstruction problem,which either consumes much memory space or requires various complicated computations.In this paper,we present a neural field(NF)-based image reconstruction scheme for BLT that uses an implicit neural representation.The proposed NFbased method establishes a transformation between the coordinate of an arbitrary spatial point and the source value of the point with a relatively light-weight multilayer perceptron,which has remarkable computational efficiency.Another simple neural network composed of two fully connected layers and a 1D convolutional layer is used to generate the neural features.Results of simulations and experiments show that the proposed NF-based method has similar performance to the photon density complement network and the two-stage network,while consuming fewer floating point operations with fewer model parameters.展开更多
In Hyperspectral Imaging(HSI),the detrimental influence of noise and distortions on data quality is profound,which has severely affected the following-on analytics and decisionmaking such as land mapping.This study pr...In Hyperspectral Imaging(HSI),the detrimental influence of noise and distortions on data quality is profound,which has severely affected the following-on analytics and decisionmaking such as land mapping.This study presents an innovative framework for assessing HSI band quality and reconstructing the low-quality bands,based on the Prophet model.By introducing a comprehensive quality metric to start,the authors approach factors in both spatial and spectral characteristics across local and global scales.This metric effectively captures the intricate noise and distortions inherent in the HSI data.Subsequently,the authors employ the Prophet model to forecast information within the low-quality bands,leveraging insights from neighbouring high-quality bands.To validate the effectiveness of the authors’proposed model,extensive experiments on three publicly available uncorrected datasets are conducted.In a head-to-head comparison,the framework against six state-ofthe-art band reconstruction algorithms including three spectral methods,two spatialspectral methods and one deep learning method is benchmarked.The authors’experiments also delve into strategies for band selection based on quality metrics and the quality evaluation of the reconstructed bands.In addition,the authors assess the classification accuracy utilising these reconstructed bands.In various experiments,the results consistently affirm the efficacy of the authors’method in HSI quality assessment and band reconstruction.Notably,the authors’approach obviates the need for manually prefiltering of noisy bands.This comprehensive framework holds promise in addressing HSI data quality concerns whilst enhancing the overall utility of HSI.展开更多
Multiple-point statistics(MPS)is a useful approach to reconstruct three-dimensional models in the macroscopic or microscopic field.Extracting spatial features for three-dimensional reconstruction from two-dimensional ...Multiple-point statistics(MPS)is a useful approach to reconstruct three-dimensional models in the macroscopic or microscopic field.Extracting spatial features for three-dimensional reconstruction from two-dimensional training images(TIs),and characterizing non-stationary features with directional ductility are two key issues in MPS simulation.This study presents a step-wise MPS-based three-dimensional structures reconstruction algorithm with the sequential process and hierarchical strategy based on two-dimensional images.An extension method is proposed to construct three-dimensional TIs.With a sequential simulation process,an initial guess at the coarsest scale is simulated,in which hierarchical strategy is used according to the characteristics of TIs.To obtain a more refined realization,an expectation-maximization like iterative process with global optimization is implemented.A concrete example of chondrite micro-structure simulation,in which one scanning electron microscopy(SEM)image of the Heyetang meteorite is used as TI,shows that the presented algorithm can simulate complex non-stationary structures.展开更多
BACKGROUND Deep learning-based super-resolution(SR)reconstruction can obtain high-quality images with more detailed information.AIM To compare multiparametric normal-resolution(NR)and SR magnetic resonance imaging(MRI...BACKGROUND Deep learning-based super-resolution(SR)reconstruction can obtain high-quality images with more detailed information.AIM To compare multiparametric normal-resolution(NR)and SR magnetic resonance imaging(MRI)in predicting the histopathologic grade in hepatocellular carcinoma.METHODS We retrospectively analyzed a total of 826 patients from two medical centers(training 459;validation 196;test 171).T2-weighted imaging,diffusion-weighted imaging,and portal venous phases were collected.Tumor segmentations were conducted automatically by 3D U-Net.Based on generative adversarial network,we utilized 3D SR reconstruction to produce SR MRI.Radiomics models were developed and validated by XGBoost and Catboost.The predictive efficiency was demonstrated by calibration curves,decision curve analysis,area under the curve(AUC)and net reclassification index(NRI).RESULTS We extracted 3045 radiomic features from both NR and SR MRI,retaining 29 and 28 features,respectively.For XGBoost models,SR MRI yielded higher AUC value than NR MRI in the validation and test cohorts(0.83 vs 0.79;0.80 vs 0.78),respectively.Consistent trends were seen in CatBoost models:SR MRI achieved AUCs of 0.89 and 0.80 compared to NR MRI’s 0.81 and 0.76.NRI indicated that the SR MRI models could improve the prediction accuracy by-1.6%to 20.9%compared to the NR MRI models.CONCLUSION Deep learning-based SR MRI could improve the predictive performance of histopathologic grade in HCC.It may be a powerful tool for better stratification management for patients with operable HCC.展开更多
In order to improve the reconstruction accuracy of magnetic resonance imaging(MRI),an accurate natural image compressed sensing(CS)reconstruction network is proposed,which combines the advantages of model-based and de...In order to improve the reconstruction accuracy of magnetic resonance imaging(MRI),an accurate natural image compressed sensing(CS)reconstruction network is proposed,which combines the advantages of model-based and deep learning-based CS-MRI methods.In theory,enhancing geometric texture details in linear reconstruction is possible.First,the optimization problem is decomposed into two problems:linear approximation and geometric compensation.Aimed at the problem of image linear approximation,the data consistency module is used to deal with it.Since the processing process will lose texture details,a neural network layer that explicitly combines image and frequency feature representation is proposed,which is named butterfly dilated geometric distillation network.The network introduces the idea of butterfly operation,skillfully integrates the features of image domain and frequency domain,and avoids the loss of texture details when extracting features in a single domain.Finally,a channel feature fusion module is designed by combining channel attention mechanism and dilated convolution.The attention of the channel makes the final output feature map focus on the more important part,thus improving the feature representation ability.The dilated convolution enlarges the receptive field,thereby obtaining more dense image feature data.The experimental results show that the peak signal-to-noise ratio of the network is 5.43 dB,5.24 dB and 3.89 dB higher than that of ISTA-Net+,FISTA and DGDN networks on the brain data set with a Cartesian sampling mask CS ratio of 10%.展开更多
Stochastic optical reconstruction microscopy(STORM),as a typical technique of single-molecule localization microscopy(SMLM),has overcome the diffraction limit by randomly switching fluorophores between fluorescent and...Stochastic optical reconstruction microscopy(STORM),as a typical technique of single-molecule localization microscopy(SMLM),has overcome the diffraction limit by randomly switching fluorophores between fluorescent and dark states,allowing for the precise localization of isolated emission patterns and the super-resolution reconstruction from millions of localized positions of single fluorophores.A critical factor influencing localization precision is the photo-switching behavior of fluorophores,which is affected by the imaging buffer.The imaging buffer typically comprises oxygen scavengers,photo-switching reagents,and refractive index regulators.Oxygen scavengers help prevent photobleaching,photo-switching reagents assist in facilitating the conversion of fluorophores,and refractive index regulators are used to adjust the refractive index of the solution.The synergistic interaction of these components promotes stable blinking of fluorophores,reduces irreversible photobleaching,and thereby ensures high-quality super-resolution imaging.This review provides a comprehensive overview of the essential compositions and functionalities of imaging buffers used in STORM,serving as a valuable resource for researchers seeking to select appropriate imaging buffers for their experiments.展开更多
Neutron capture event imaging is a novel technique that has the potential to substantially enhance the resolution of existing imaging systems.This study provides a measurement method for neutron capture event distribu...Neutron capture event imaging is a novel technique that has the potential to substantially enhance the resolution of existing imaging systems.This study provides a measurement method for neutron capture event distribution along with multiple reconstruction methods for super-resolution imaging.The proposed technology reduces the point-spread function of an imag-ing system through single-neutron detection and event reconstruction,thereby significantly improving imaging resolution.A single-neutron detection experiment was conducted using a highly practical and efficient^(6)LiF-ZnS scintillation screen of a cold neutron imaging device in the research reactor.In milliseconds of exposure time,a large number of weak light clusters and their distribution in the scintillation screen were recorded frame by frame,to complete single-neutron detection.Several reconstruction algorithms were proposed for the calculations.The location of neutron capture was calculated using several processing methods such as noise removal,filtering,spot segmentation,contour analysis,and local positioning.The proposed algorithm achieved a higher imaging resolution and faster reconstruction speed,and single-neutron super-resolution imaging was realized by combining single-neutron detection experiments and reconstruction calculations.The results show that the resolution of the 100μm thick^(6)LiF-ZnS scintillation screen can be improved from 125 to 40 microns.This indicates that the proposed single-neutron detection and calculation method is effective and can significantly improve imaging resolution.展开更多
This study systematically investigated the Lean Blowoff(LBO)limits of Two-Dimensional(2D)bluff-body stabilized premixed flames by varying the air mass flow rate,inflowtemperature,bluff-body width,and fuel type.The dat...This study systematically investigated the Lean Blowoff(LBO)limits of Two-Dimensional(2D)bluff-body stabilized premixed flames by varying the air mass flow rate,inflowtemperature,bluff-body width,and fuel type.The data of LBO limits were analyzed and fittedaccording to the Damk?hler(Da)and Reynolds(Re)numbers,and the fitting accuracy of LBO datawas highly improved by a modified characteristic length simultaneously considering the length andwidth of the bluff body,which is usually neglected in the previous studies.Moreover,to our knowl-edge,this is the first time that simultaneous transverse and spanwise OH*-Chemiluminescence(CL)imaging has been performed to examine the three-dimensional behavior of the LBO process.The flame stability is heavily affected by the mass and energy transport between reactants andproducts in both directions,potentially leading to the flame pinch-off.The intensity and positionof the upstream flame after pinch-off are decisive to the occurrence of the following LBO.Whenthe upstream flame after pinch-off is weak and close to the bluff body,it cannot re-ignite thedownstream unburnt gas.Subsequently,a permanent downstream extinction occurs,and theLBO takes place.The results help understand the LBO mechanism of 2D bluff-body stabilizedflames.展开更多
Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging du...Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging due to advances in both camouflage materials and biological mimicry.Although multispectral-RGB based technology shows promise,conventional dual-aperture multispectral-RGB imaging systems are constrained by imprecise and time-consuming registration and fusion across different modalities,limiting their performance.Here,we propose the Reconstructed Multispectral-RGB Fusion Network(RMRF-Net),which reconstructs RGB images into multispectral ones,enabling efficient multimodal segmentation using only an RGB camera.Specifically,RMRF-Net employs a divergentsimilarity feature correction strategy to minimize reconstruction errors and includes an efficient boundary-aware decoder to enhance object contours.Notably,we establish the first real-world aerial multispectral-RGB semantic segmentation of camouflage objects dataset,including 11 object categories.Experimental results demonstrate that RMRF-Net outperforms existing methods,achieving 17.38 FPS on the NVIDIA Jetson AGX Orin,with only a 0.96%drop in mIoU compared to the RTX 3090,showing its practical applicability in multimodal remote sensing.展开更多
Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditiona...Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.展开更多
Three-dimensional(3D)reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units.In the coming years,most patient care will shift toward this new p...Three-dimensional(3D)reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units.In the coming years,most patient care will shift toward this new paradigm.However,development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved,most of which are dependent on human expertise.In this review,a survey of pre-processing steps was conducted,and reconstruction techniques for several organs in medical diagnosis were studied.Various methods and principles related to 3D reconstruction were highlighted.The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.展开更多
In this paper,we introduce an accelerating algorithm based on the Taylor series for reconstructing target images in the spectral digital image correlation method(SDIC).The Taylor series image reconstruction method is ...In this paper,we introduce an accelerating algorithm based on the Taylor series for reconstructing target images in the spectral digital image correlation method(SDIC).The Taylor series image reconstruction method is employed instead of the previous direct Fourier transform(DFT)image reconstruction method,which consumes the majority of the computational time for target image reconstruction.The partial derivatives in the Taylor series are computed using the fast Fourier transform(FFT)of the entire image,following the principles of Fourier transform theory.To examine the impact of different orders of Taylor series expansion on accuracy and efficiency,we employ third-and fourth-order Taylor series image reconstruction methods and compare them with the DFT image reconstruction method through simulated experiments.As a result of these enhancements,the computational efficiency using the third-and fourth-order Taylor series improves by factors of 57 and 46,respectively,compared to the previous method.In terms of analysis accuracy,within a strain range of 0–0.1 and without the addition of image noise,the accuracy of the proposed method increases with higher expansion orders,surpassing that of the DFT image reconstruction method when the fourth order is utilized.However,when different levels of Gaussian noise are applied to simulated images individually,the accuracy of the third-or fourth-order Taylor series expansion method is superior to that of the DFT reconstruction method.Finally,we present the analyzed experimental results of a silicone rubber plate specimen with bilateral cracks under uniaxial tension.展开更多
Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high...Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high contrast.However,limited by the equipment cost and reconstruction time requirements,the existing PAI systems distributed with annular array transducers are difficult to take into account both the image quality and the imaging speed.In this paper,a triple-path feature transform network(TFT-Net)for ring-array photoacoustic tomography is proposed to enhance the imaging quality from limited-view and sparse measurement data.Specifically,the network combines the raw photoacoustic pressure signals and conventional linear reconstruction images as input data,and takes the photoacoustic physical model as a prior information to guide the reconstruction process.In addition,to enhance the ability of extracting signal features,the residual block and squeeze and excitation block are introduced into the TFT-Net.For further efficient reconstruction,the final output of photoacoustic signals uses‘filter-then-upsample’operation with a pixel-shuffle multiplexer and a max out module.Experiment results on simulated and in-vivo data demonstrate that the constructed TFT-Net can restore the target boundary clearly,reduce background noise,and realize fast and high-quality photoacoustic image reconstruction of limited view with sparse sampling.展开更多
Aiming at the problems such as low reconstruction efficiency,fuzzy texture details,and difficult convergence of reconstruction network face image super-resolution reconstruction algorithms,a new super-resolution recon...Aiming at the problems such as low reconstruction efficiency,fuzzy texture details,and difficult convergence of reconstruction network face image super-resolution reconstruction algorithms,a new super-resolution reconstruction algorithm with residual concern was proposed.Firstly,to solve the influence of redundant and invalid information about the face image super-resolution reconstruction network,an attention mechanism was introduced into the feature extraction module of the network,which improved the feature utilization rate of the overall network.Secondly,to alleviate the problem of gradient disappearance,the adaptive residual was introduced into the network to make the network model easier to converge during training,and features were supplemented according to the needs during training.The experimental results showed that the proposed algorithm had better reconstruction performance,more facial details,and clearer texture in the reconstructed face image than the comparison algorithm.In objective evaluation,the proposed algorithm's peak signalto-noise ratio and structural similarity were also better than other algorithms.展开更多
Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the thr...Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.展开更多
In order to solve the problem of the lack of ornamental value and research value of ancient mural paintings due to low resolution and fuzzy texture details,a super resolution(SR)method based on generative adduction ne...In order to solve the problem of the lack of ornamental value and research value of ancient mural paintings due to low resolution and fuzzy texture details,a super resolution(SR)method based on generative adduction network(GAN)was proposed.This method reconstructed the detail texture of mural image better.Firstly,in view of the insufficient utilization of shallow image features,information distillation blocks(IDB)were introduced to extract shallow image features and enhance the output results of the network behind.Secondly,residual dense blocks with residual scaling and feature fusion(RRDB-Fs)were used to extract deep image features,which removed the BN layer in the residual block that affected the quality of image generation,and improved the training speed of the network.Furthermore,local feature fusion and global feature fusion were applied in the generation network,and the features of different levels were merged together adaptively,so that the reconstructed image contained rich details.Finally,in calculating the perceptual loss,the brightness consistency between the reconstructed fresco and the original fresco was enhanced by using the features before activation,while avoiding artificial interference.The experimental results showed that the peak signal-to-noise ratio and structural similarity metrics were improved compared with other algorithms,with an improvement of 0.512 dB-3.016 dB in peak signal-to-noise ratio and 0.009-0.089 in structural similarity,and the proposed method had better visual effects.展开更多
3D reconstruction plays an increasingly important role in modern photogrammetric systems.Conventional satellite or aerial-based remote sensing(RS)platforms can provide the necessary data sources for the 3D reconstruct...3D reconstruction plays an increasingly important role in modern photogrammetric systems.Conventional satellite or aerial-based remote sensing(RS)platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities.Even with low-altitude Unmanned Aerial Vehicles(UAVs),3D reconstruction in complicated situations,such as urban canyons and indoor scenes,is challenging due to frequent tracking failures between camera frames and high data collection costs.Recently,spherical images have been extensively used due to the capability of recording surrounding environments from one image.In contrast to perspective images with limited Field of View(FOV),spherical images can cover the whole scene with full horizontal and vertical FOV and facilitate camera tracking and data acquisition in these complex scenes.With the rapid evolution and extensive use of professional and consumer-grade spherical cameras,spherical images show great potential for the 3D modeling of urban and indoor scenes.Classical 3D reconstruction pipelines,however,cannot be directly used for spherical images.Besides,there exist few software packages that are designed for the 3D reconstruction from spherical images.As a result,this research provides a thorough survey of the state-of-the-art for 3D reconstruction from spherical images in terms of data acquisition,feature detection and matching,image orientation,and dense matching as well as presenting promising applications and discussing potential prospects.We anticipate that this study offers insightful clues to direct future research.展开更多
文摘Image super-resolution reconstruction technology is currently widely used in medical imaging,video surveillance,and industrial quality inspection.It not only enhances image quality but also improves details and visual perception,significantly increasing the utility of low-resolution images.In this study,an improved image superresolution reconstruction model based on Generative Adversarial Networks(SRGAN)was proposed.This model introduced a channel and spatial attention mechanism(CSAB)in the generator,allowing it to effectively leverage the information from the input image to enhance feature representations and capture important details.The discriminator was designed with an improved PatchGAN architecture,which more accurately captured local details and texture information of the image.With these enhanced generator and discriminator architectures and an optimized loss function design,this method demonstrated superior performance in image quality assessment metrics.Experimental results showed that this model outperforms traditional methods,presenting more detailed and realistic image details in the visual effects.
基金supported by the Interdisciplinary project of Dalian University DLUXK-2023-ZD-001.
文摘As a form of discrete representation learning,Vector Quantized Variational Autoencoders(VQ-VAE)have increasingly been applied to generative and multimodal tasks due to their ease of embedding and representative capacity.However,existing VQ-VAEs often perform quantization in the spatial domain,ignoring global structural information and potentially suffering from codebook collapse and information coupling issues.This paper proposes a frequency quantized variational autoencoder(FQ-VAE)to address these issues.The proposed method transforms image features into linear combinations in the frequency domain using a 2D fast Fourier transform(2D-FFT)and performs adaptive quantization on these frequency components to preserve image’s global relationships.The codebook is dynamically optimized to avoid collapse and information coupling issue by considering the usage frequency and dependency of code vectors.Furthermore,we introduce a post-processing module based on graph convolutional networks to further improve reconstruction quality.Experimental results on four public datasets demonstrate that the proposed method outperforms state-of-the-art approaches in terms of Structural Similarity Index(SSIM),Learned Perceptual Image Patch Similarity(LPIPS),and Reconstruction Fréchet Inception Distance(rFID).In the experiments on the CIFAR-10 dataset,compared to the baselinemethod VQ-VAE,the proposedmethod improves the abovemetrics by 4.9%,36.4%,and 52.8%,respectively.
文摘Among hyperspectral imaging technologies, interferometric spectral imaging is widely used in remote sening due to advantages of large luminous flux and high resolution. However, with complicated mechanism, interferometric imaging faces the impact of multi-stage degradation. Most exsiting interferometric spectrum reconstruction methods are based on tradition model-based framework with multiple steps, showing poor efficiency and restricted performance. Thus, we propose an interferometric spectrum reconstruction method based on degradation synthesis and deep learning.Firstly, based on imaging mechanism, we proposed an mathematical model of interferometric imaging to analyse the degradation components as noises and trends during imaging. The model consists of three stages, namely instrument degradation, sensing degradation, and signal-independent degradation process. Then, we designed calibration-based method to estimate parameters in the model, of which the results are used for synthesizing realistic dataset for learning-based algorithms.In addition, we proposed a dual-stage interferogram spectrum reconstruction framework, which supports pre-training and integration of denoising DNNs. Experiments exhibits the reliability of our degradation model and synthesized data, and the effectiveness of the proposed reconstruction method.
基金supported in part by the National Natural Science Foundation of China(62101278,62001379,62271023)Beijing Natural Science Foundation(7242269).
文摘Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic imaging techniques,such as bioluminescence tomography(BLT).Nevertheless,nearly every existing DL-based method utilizes an explicit neural representation for the reconstruction problem,which either consumes much memory space or requires various complicated computations.In this paper,we present a neural field(NF)-based image reconstruction scheme for BLT that uses an implicit neural representation.The proposed NFbased method establishes a transformation between the coordinate of an arbitrary spatial point and the source value of the point with a relatively light-weight multilayer perceptron,which has remarkable computational efficiency.Another simple neural network composed of two fully connected layers and a 1D convolutional layer is used to generate the neural features.Results of simulations and experiments show that the proposed NF-based method has similar performance to the photon density complement network and the two-stage network,while consuming fewer floating point operations with fewer model parameters.
基金National Natural Science Foundation Major Project of China,Grant/Award Number:42192580Guangdong Province Key Construction Discipline Scientific Research Ability Promotion Project,Grant/Award Number:2022ZDJS015。
文摘In Hyperspectral Imaging(HSI),the detrimental influence of noise and distortions on data quality is profound,which has severely affected the following-on analytics and decisionmaking such as land mapping.This study presents an innovative framework for assessing HSI band quality and reconstructing the low-quality bands,based on the Prophet model.By introducing a comprehensive quality metric to start,the authors approach factors in both spatial and spectral characteristics across local and global scales.This metric effectively captures the intricate noise and distortions inherent in the HSI data.Subsequently,the authors employ the Prophet model to forecast information within the low-quality bands,leveraging insights from neighbouring high-quality bands.To validate the effectiveness of the authors’proposed model,extensive experiments on three publicly available uncorrected datasets are conducted.In a head-to-head comparison,the framework against six state-ofthe-art band reconstruction algorithms including three spectral methods,two spatialspectral methods and one deep learning method is benchmarked.The authors’experiments also delve into strategies for band selection based on quality metrics and the quality evaluation of the reconstructed bands.In addition,the authors assess the classification accuracy utilising these reconstructed bands.In various experiments,the results consistently affirm the efficacy of the authors’method in HSI quality assessment and band reconstruction.Notably,the authors’approach obviates the need for manually prefiltering of noisy bands.This comprehensive framework holds promise in addressing HSI data quality concerns whilst enhancing the overall utility of HSI.
基金substantially supported by the National Natural Science Foundation of China(NSFC)Program(Nos.41972302,41772345)。
文摘Multiple-point statistics(MPS)is a useful approach to reconstruct three-dimensional models in the macroscopic or microscopic field.Extracting spatial features for three-dimensional reconstruction from two-dimensional training images(TIs),and characterizing non-stationary features with directional ductility are two key issues in MPS simulation.This study presents a step-wise MPS-based three-dimensional structures reconstruction algorithm with the sequential process and hierarchical strategy based on two-dimensional images.An extension method is proposed to construct three-dimensional TIs.With a sequential simulation process,an initial guess at the coarsest scale is simulated,in which hierarchical strategy is used according to the characteristics of TIs.To obtain a more refined realization,an expectation-maximization like iterative process with global optimization is implemented.A concrete example of chondrite micro-structure simulation,in which one scanning electron microscopy(SEM)image of the Heyetang meteorite is used as TI,shows that the presented algorithm can simulate complex non-stationary structures.
基金Supported by AI+Health Collaborative Innovation Cultivation Project of Beijing City,No.Z221100003522005.
文摘BACKGROUND Deep learning-based super-resolution(SR)reconstruction can obtain high-quality images with more detailed information.AIM To compare multiparametric normal-resolution(NR)and SR magnetic resonance imaging(MRI)in predicting the histopathologic grade in hepatocellular carcinoma.METHODS We retrospectively analyzed a total of 826 patients from two medical centers(training 459;validation 196;test 171).T2-weighted imaging,diffusion-weighted imaging,and portal venous phases were collected.Tumor segmentations were conducted automatically by 3D U-Net.Based on generative adversarial network,we utilized 3D SR reconstruction to produce SR MRI.Radiomics models were developed and validated by XGBoost and Catboost.The predictive efficiency was demonstrated by calibration curves,decision curve analysis,area under the curve(AUC)and net reclassification index(NRI).RESULTS We extracted 3045 radiomic features from both NR and SR MRI,retaining 29 and 28 features,respectively.For XGBoost models,SR MRI yielded higher AUC value than NR MRI in the validation and test cohorts(0.83 vs 0.79;0.80 vs 0.78),respectively.Consistent trends were seen in CatBoost models:SR MRI achieved AUCs of 0.89 and 0.80 compared to NR MRI’s 0.81 and 0.76.NRI indicated that the SR MRI models could improve the prediction accuracy by-1.6%to 20.9%compared to the NR MRI models.CONCLUSION Deep learning-based SR MRI could improve the predictive performance of histopathologic grade in HCC.It may be a powerful tool for better stratification management for patients with operable HCC.
基金the National Natural Science Foundation of China(No.61962032)。
文摘In order to improve the reconstruction accuracy of magnetic resonance imaging(MRI),an accurate natural image compressed sensing(CS)reconstruction network is proposed,which combines the advantages of model-based and deep learning-based CS-MRI methods.In theory,enhancing geometric texture details in linear reconstruction is possible.First,the optimization problem is decomposed into two problems:linear approximation and geometric compensation.Aimed at the problem of image linear approximation,the data consistency module is used to deal with it.Since the processing process will lose texture details,a neural network layer that explicitly combines image and frequency feature representation is proposed,which is named butterfly dilated geometric distillation network.The network introduces the idea of butterfly operation,skillfully integrates the features of image domain and frequency domain,and avoids the loss of texture details when extracting features in a single domain.Finally,a channel feature fusion module is designed by combining channel attention mechanism and dilated convolution.The attention of the channel makes the final output feature map focus on the more important part,thus improving the feature representation ability.The dilated convolution enlarges the receptive field,thereby obtaining more dense image feature data.The experimental results show that the peak signal-to-noise ratio of the network is 5.43 dB,5.24 dB and 3.89 dB higher than that of ISTA-Net+,FISTA and DGDN networks on the brain data set with a Cartesian sampling mask CS ratio of 10%.
基金funded by the National Natural Science Foundation of China(No.62305041)the Natural Science Foundation of Liaoning Province(No.2023-MS-103)。
文摘Stochastic optical reconstruction microscopy(STORM),as a typical technique of single-molecule localization microscopy(SMLM),has overcome the diffraction limit by randomly switching fluorophores between fluorescent and dark states,allowing for the precise localization of isolated emission patterns and the super-resolution reconstruction from millions of localized positions of single fluorophores.A critical factor influencing localization precision is the photo-switching behavior of fluorophores,which is affected by the imaging buffer.The imaging buffer typically comprises oxygen scavengers,photo-switching reagents,and refractive index regulators.Oxygen scavengers help prevent photobleaching,photo-switching reagents assist in facilitating the conversion of fluorophores,and refractive index regulators are used to adjust the refractive index of the solution.The synergistic interaction of these components promotes stable blinking of fluorophores,reduces irreversible photobleaching,and thereby ensures high-quality super-resolution imaging.This review provides a comprehensive overview of the essential compositions and functionalities of imaging buffers used in STORM,serving as a valuable resource for researchers seeking to select appropriate imaging buffers for their experiments.
基金supported by the National Natural Science Foundation of China(Nos.12205271,12075217,U20B2011,and 51978218)Sichuan Science and Technology Program(No.2019ZDZX0010)the National Key R&D Program of China(No.2022YFA1604002).
文摘Neutron capture event imaging is a novel technique that has the potential to substantially enhance the resolution of existing imaging systems.This study provides a measurement method for neutron capture event distribution along with multiple reconstruction methods for super-resolution imaging.The proposed technology reduces the point-spread function of an imag-ing system through single-neutron detection and event reconstruction,thereby significantly improving imaging resolution.A single-neutron detection experiment was conducted using a highly practical and efficient^(6)LiF-ZnS scintillation screen of a cold neutron imaging device in the research reactor.In milliseconds of exposure time,a large number of weak light clusters and their distribution in the scintillation screen were recorded frame by frame,to complete single-neutron detection.Several reconstruction algorithms were proposed for the calculations.The location of neutron capture was calculated using several processing methods such as noise removal,filtering,spot segmentation,contour analysis,and local positioning.The proposed algorithm achieved a higher imaging resolution and faster reconstruction speed,and single-neutron super-resolution imaging was realized by combining single-neutron detection experiments and reconstruction calculations.The results show that the resolution of the 100μm thick^(6)LiF-ZnS scintillation screen can be improved from 125 to 40 microns.This indicates that the proposed single-neutron detection and calculation method is effective and can significantly improve imaging resolution.
基金the financial support of the National Natural Science Foundation of China(Nos.U2141221 and 52076136)the National Science and Technology Major Project,China(Nos.J2019-Ⅲ-0004-0047 and Y2022-Ⅲ-0001-0010)+1 种基金the Center for Basic Science of Aero Engines and Gas Turbines Project,China(No.P2022-B-Ⅱ019-001)the Natural Science Foundation of Shanghai,China(Nos.22ZR1467900 and 23ZR1481400)。
文摘This study systematically investigated the Lean Blowoff(LBO)limits of Two-Dimensional(2D)bluff-body stabilized premixed flames by varying the air mass flow rate,inflowtemperature,bluff-body width,and fuel type.The data of LBO limits were analyzed and fittedaccording to the Damk?hler(Da)and Reynolds(Re)numbers,and the fitting accuracy of LBO datawas highly improved by a modified characteristic length simultaneously considering the length andwidth of the bluff body,which is usually neglected in the previous studies.Moreover,to our knowl-edge,this is the first time that simultaneous transverse and spanwise OH*-Chemiluminescence(CL)imaging has been performed to examine the three-dimensional behavior of the LBO process.The flame stability is heavily affected by the mass and energy transport between reactants andproducts in both directions,potentially leading to the flame pinch-off.The intensity and positionof the upstream flame after pinch-off are decisive to the occurrence of the following LBO.Whenthe upstream flame after pinch-off is weak and close to the bluff body,it cannot re-ignite thedownstream unburnt gas.Subsequently,a permanent downstream extinction occurs,and theLBO takes place.The results help understand the LBO mechanism of 2D bluff-body stabilizedflames.
基金National Natural Science Foundation of China(Grant Nos.62005049 and 62072110)Natural Science Foundation of Fujian Province(Grant No.2020J01451).
文摘Accurate segmentation of camouflage objects in aerial imagery is vital for improving the efficiency of UAV-based reconnaissance and rescue missions.However,camouflage object segmentation is increasingly challenging due to advances in both camouflage materials and biological mimicry.Although multispectral-RGB based technology shows promise,conventional dual-aperture multispectral-RGB imaging systems are constrained by imprecise and time-consuming registration and fusion across different modalities,limiting their performance.Here,we propose the Reconstructed Multispectral-RGB Fusion Network(RMRF-Net),which reconstructs RGB images into multispectral ones,enabling efficient multimodal segmentation using only an RGB camera.Specifically,RMRF-Net employs a divergentsimilarity feature correction strategy to minimize reconstruction errors and includes an efficient boundary-aware decoder to enhance object contours.Notably,we establish the first real-world aerial multispectral-RGB semantic segmentation of camouflage objects dataset,including 11 object categories.Experimental results demonstrate that RMRF-Net outperforms existing methods,achieving 17.38 FPS on the NVIDIA Jetson AGX Orin,with only a 0.96%drop in mIoU compared to the RTX 3090,showing its practical applicability in multimodal remote sensing.
基金funded by the National Natural Science Foundation of China(62125504,61827825,and 31901059)Zhejiang Provincial Ten Thousand Plan for Young Top Talents(2020R52001)Open Project Program of Wuhan National Laboratory for Optoelectronics(2021WNLOKF007).
文摘Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.
文摘Three-dimensional(3D)reconstruction of human organs has gained attention in recent years due to advances in the Internet and graphics processing units.In the coming years,most patient care will shift toward this new paradigm.However,development of fast and accurate 3D models from medical images or a set of medical scans remains a daunting task due to the number of pre-processing steps involved,most of which are dependent on human expertise.In this review,a survey of pre-processing steps was conducted,and reconstruction techniques for several organs in medical diagnosis were studied.Various methods and principles related to 3D reconstruction were highlighted.The usefulness of 3D reconstruction of organs in medical diagnosis was also highlighted.
基金supported by the National Natural Science Foundation of China(Grant Nos.12272145 and 11972013)the Ministry of Science and Technology of China(Grant No.2018YFF01014200)Hubei Provincial Natural Science Foundation of China(Grant No.2022CFB288).
文摘In this paper,we introduce an accelerating algorithm based on the Taylor series for reconstructing target images in the spectral digital image correlation method(SDIC).The Taylor series image reconstruction method is employed instead of the previous direct Fourier transform(DFT)image reconstruction method,which consumes the majority of the computational time for target image reconstruction.The partial derivatives in the Taylor series are computed using the fast Fourier transform(FFT)of the entire image,following the principles of Fourier transform theory.To examine the impact of different orders of Taylor series expansion on accuracy and efficiency,we employ third-and fourth-order Taylor series image reconstruction methods and compare them with the DFT image reconstruction method through simulated experiments.As a result of these enhancements,the computational efficiency using the third-and fourth-order Taylor series improves by factors of 57 and 46,respectively,compared to the previous method.In terms of analysis accuracy,within a strain range of 0–0.1 and without the addition of image noise,the accuracy of the proposed method increases with higher expansion orders,surpassing that of the DFT image reconstruction method when the fourth order is utilized.However,when different levels of Gaussian noise are applied to simulated images individually,the accuracy of the third-or fourth-order Taylor series expansion method is superior to that of the DFT reconstruction method.Finally,we present the analyzed experimental results of a silicone rubber plate specimen with bilateral cracks under uniaxial tension.
基金supported by National Key R&D Program of China[2022YFC2402400]the National Natural Science Foundation of China[Grant No.62275062]Guangdong Provincial Key Laboratory of Biomedical Optical Imaging Technology[Grant No.2020B121201010-4].
文摘Photoacoustic imaging(PAI)is a noninvasive emerging imaging method based on the photoacoustic effect,which provides necessary assistance for medical diagnosis.It has the characteristics of large imaging depth and high contrast.However,limited by the equipment cost and reconstruction time requirements,the existing PAI systems distributed with annular array transducers are difficult to take into account both the image quality and the imaging speed.In this paper,a triple-path feature transform network(TFT-Net)for ring-array photoacoustic tomography is proposed to enhance the imaging quality from limited-view and sparse measurement data.Specifically,the network combines the raw photoacoustic pressure signals and conventional linear reconstruction images as input data,and takes the photoacoustic physical model as a prior information to guide the reconstruction process.In addition,to enhance the ability of extracting signal features,the residual block and squeeze and excitation block are introduced into the TFT-Net.For further efficient reconstruction,the final output of photoacoustic signals uses‘filter-then-upsample’operation with a pixel-shuffle multiplexer and a max out module.Experiment results on simulated and in-vivo data demonstrate that the constructed TFT-Net can restore the target boundary clearly,reduce background noise,and realize fast and high-quality photoacoustic image reconstruction of limited view with sparse sampling.
基金supported by National Natural Science Foundation of China(No.62063014)。
文摘Aiming at the problems such as low reconstruction efficiency,fuzzy texture details,and difficult convergence of reconstruction network face image super-resolution reconstruction algorithms,a new super-resolution reconstruction algorithm with residual concern was proposed.Firstly,to solve the influence of redundant and invalid information about the face image super-resolution reconstruction network,an attention mechanism was introduced into the feature extraction module of the network,which improved the feature utilization rate of the overall network.Secondly,to alleviate the problem of gradient disappearance,the adaptive residual was introduced into the network to make the network model easier to converge during training,and features were supplemented according to the needs during training.The experimental results showed that the proposed algorithm had better reconstruction performance,more facial details,and clearer texture in the reconstructed face image than the comparison algorithm.In objective evaluation,the proposed algorithm's peak signalto-noise ratio and structural similarity were also better than other algorithms.
基金Supported by the Key R&D Projects in Shaanxi Province(2022JBGS3-08)。
文摘Rapidly and accurately assessing the geometric characteristics of coarse aggregate particles is crucial for ensuring pavement performance in highway engineering.This article introduces an innovative system for the three-dimensional(3D)surface reconstruction of coarse aggregate particles using occlusion-free multi-view imaging.The system captures synchronized images of particles in free fall,employing a matte sphere and a nonlinear optimization approach to estimate the camera projection matrices.A pre-trained segmentation model is utilized to eliminate the background of the images.The Shape from Silhouettes(SfS)algorithm is then applied to generate 3D voxel data,followed by the Marching Cubes algorithm to construct the 3D surface contour.Validation against standard parts and diverse coarse aggregate particles confirms the method's high accuracy,with an average measurement precision of 0.434 mm and a significant increase in scanning and reconstruction efficiency.
文摘In order to solve the problem of the lack of ornamental value and research value of ancient mural paintings due to low resolution and fuzzy texture details,a super resolution(SR)method based on generative adduction network(GAN)was proposed.This method reconstructed the detail texture of mural image better.Firstly,in view of the insufficient utilization of shallow image features,information distillation blocks(IDB)were introduced to extract shallow image features and enhance the output results of the network behind.Secondly,residual dense blocks with residual scaling and feature fusion(RRDB-Fs)were used to extract deep image features,which removed the BN layer in the residual block that affected the quality of image generation,and improved the training speed of the network.Furthermore,local feature fusion and global feature fusion were applied in the generation network,and the features of different levels were merged together adaptively,so that the reconstructed image contained rich details.Finally,in calculating the perceptual loss,the brightness consistency between the reconstructed fresco and the original fresco was enhanced by using the features before activation,while avoiding artificial interference.The experimental results showed that the peak signal-to-noise ratio and structural similarity metrics were improved compared with other algorithms,with an improvement of 0.512 dB-3.016 dB in peak signal-to-noise ratio and 0.009-0.089 in structural similarity,and the proposed method had better visual effects.
基金funded by the National Natural Science Foundation of China[Grant No.42371442]the Hubei Provincial Natural Science Foundation of China[Grant No.2023AFB568]+1 种基金the Hong Kong Scholars Program[Grant No.2021-114]the Open Research fund from the Hubei Luojia Laboratory[Grand No.230100013].
文摘3D reconstruction plays an increasingly important role in modern photogrammetric systems.Conventional satellite or aerial-based remote sensing(RS)platforms can provide the necessary data sources for the 3D reconstruction of large-scale landforms and cities.Even with low-altitude Unmanned Aerial Vehicles(UAVs),3D reconstruction in complicated situations,such as urban canyons and indoor scenes,is challenging due to frequent tracking failures between camera frames and high data collection costs.Recently,spherical images have been extensively used due to the capability of recording surrounding environments from one image.In contrast to perspective images with limited Field of View(FOV),spherical images can cover the whole scene with full horizontal and vertical FOV and facilitate camera tracking and data acquisition in these complex scenes.With the rapid evolution and extensive use of professional and consumer-grade spherical cameras,spherical images show great potential for the 3D modeling of urban and indoor scenes.Classical 3D reconstruction pipelines,however,cannot be directly used for spherical images.Besides,there exist few software packages that are designed for the 3D reconstruction from spherical images.As a result,this research provides a thorough survey of the state-of-the-art for 3D reconstruction from spherical images in terms of data acquisition,feature detection and matching,image orientation,and dense matching as well as presenting promising applications and discussing potential prospects.We anticipate that this study offers insightful clues to direct future research.
