Existing Transformer-based image captioning models typically rely on the self-attention mechanism to capture long-range dependencies,which effectively extracts and leverages the global correlation of image features.Ho...Existing Transformer-based image captioning models typically rely on the self-attention mechanism to capture long-range dependencies,which effectively extracts and leverages the global correlation of image features.However,these models still face challenges in effectively capturing local associations.Moreover,since the encoder extracts global and local association features that focus on different semantic information,semantic noise may occur during the decoding stage.To address these issues,we propose the Local Relationship Enhanced Gated Transformer(LREGT).In the encoder part,we introduce the Local Relationship Enhanced Encoder(LREE),whose core component is the Local Relationship Enhanced Module(LREM).LREM consists of two novel designs:the Local Correlation Perception Module(LCPM)and the Local-Global Fusion Module(LGFM),which are beneficial for generating a comprehensive feature representation that integrates both global and local information.In the decoder part,we propose the Dual-level Multi-branch Gated Decoder(DMGD).It first creates multiple decoding branches to generate multi-perspective contextual feature representations.Subsequently,it employs the Dual-Level Gating Mechanism(DLGM)to model the multi-level relationships of these multi-perspective contextual features,enhancing their fine-grained semantics and intrinsic relationship representations.This ultimately leads to the generation of high-quality and semantically rich image captions.Experiments on the standard MSCOCO dataset demonstrate that LREGT achieves state-of-the-art performance,with a CIDEr score of 140.8 and BLEU-4 score of 41.3,significantly outperforming existing mainstream methods.These results highlight LREGT’s superiority in capturing complex visual relationships and resolving semantic noise during decoding.展开更多
Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to ...Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to the inability to effectively capture global information from images,CNNs can easily lead to loss of contours and textures in segmentation results.Notice that the transformer model can effectively capture the properties of long-range dependencies in the image,and furthermore,combining the CNN and the transformer can effectively extract local details and global contextual features of the image.Motivated by this,we propose a multi-branch and multi-scale attention network(M2ANet)for medical image segmentation,whose architecture consists of three components.Specifically,in the first component,we construct an adaptive multi-branch patch module for parallel extraction of image features to reduce information loss caused by downsampling.In the second component,we apply residual block to the well-known convolutional block attention module to enhance the network’s ability to recognize important features of images and alleviate the phenomenon of gradient vanishing.In the third component,we design a multi-scale feature fusion module,in which we adopt adaptive average pooling and position encoding to enhance contextual features,and then multi-head attention is introduced to further enrich feature representation.Finally,we validate the effectiveness and feasibility of the proposed M2ANet method through comparative experiments on four benchmark medical image segmentation datasets,particularly in the context of preserving contours and textures.展开更多
Objective:In the Radiology Department of Mzuzu Central Hospital(MCH),daily training for radiographers now includes content on Computed Tomography(CT)image quality control and equipment maintenance to ensure the normal...Objective:In the Radiology Department of Mzuzu Central Hospital(MCH),daily training for radiographers now includes content on Computed Tomography(CT)image quality control and equipment maintenance to ensure the normal,continuous,and stable operation of the 16-slice spiral CT scanner.Methods:Through comprehensive analysis of relevant equipment,we have identified key parameters that significantly impact CT image quality.Innovative optimization strategies and solutions targeting these parameters have been developed and integrated into daily training programs.Furthermore,starting from an examination of prevalent failure modes observed in CT equipment,we delve into essential maintenance and preservation techniques that CT technologists must master to ensure optimal system performance.Results:(1)Crucial factors affecting CT image quality include artifacts,noise,partial volume effects,and surrounding gap phenomena,alongside spatial and density resolutions,CT dose,reconstruction algorithms,and human factors during the scanning process.In the daily training for radiographers,emphasis is placed on strictly implementing image quality control measures at every stage of the CT scanning process and skillfully applying advanced scanning and image processing techniques.By doing so,we can provide clinicians with accurate and reliable imaging references for diagnosis and treatment.(2)Strategies for CT equipment maintenance:①Environmental inspection of the CT room to ensure cleanliness and hygiene.②Rational and accurate operation,including calibration software proficiency.③Regular maintenance and servicing for minimizing machine downtime.④Maintenance of the CT X-ray tube.CT technicians can become proficient in equipment maintenance and upkeep techniques through training,which can significantly extend the service life of CT systems and reduce the occurrence of malfunctions.Conclusion:Through the regular implementation of rigorous CT image quality control training for radiology technicians,coupled with diligent and proactive CT equipment maintenance,we have observed profound and beneficial impacts on improving image quality.The accuracy and fidelity of radiological data ultimately leads to more accurate diagnoses and effective treatments.展开更多
To improve image quality under low illumination conditions,a novel low-light image enhancement method is proposed in this paper based on multi-illumination estimation and multi-scale fusion(MIMS).Firstly,the illuminat...To improve image quality under low illumination conditions,a novel low-light image enhancement method is proposed in this paper based on multi-illumination estimation and multi-scale fusion(MIMS).Firstly,the illumination is processed by contrast-limited adaptive histogram equalization(CLAHE),adaptive complementary gamma function(ACG),and adaptive detail preserving S-curve(ADPS),respectively,to obtain three components.Then,the fusion-relevant features,exposure,and color contrast are selected as the weight maps.Subsequently,these components and weight maps are fused through multi-scale to generate enhanced illumination.Finally,the enhanced images are obtained by multiplying the enhanced illumination and reflectance.Compared with existing approaches,this proposed method achieves an average increase of 0.81%and 2.89%in the structural similarity index measurement(SSIM)and peak signal-to-noise ratio(PSNR),and a decrease of 6.17%and 32.61%in the natural image quality evaluator(NIQE)and gradient magnitude similarity deviation(GMSD),respectively.展开更多
Although guided image filtering(GIF) is known for preserving edges and fast computation,it may produce inaccurate outputs in depth map restoration.In this paper,a novel confidence-weighted GIF called mutual-structure ...Although guided image filtering(GIF) is known for preserving edges and fast computation,it may produce inaccurate outputs in depth map restoration.In this paper,a novel confidence-weighted GIF called mutual-structure weighted GIF(MSWGIF) is proposed,which replaces the mean filtering strategy in GIF during handling overlapping windows.The confidence value is composed of a depth term and a mutual-structure term,where the depth term is utilized to protect the edges of the output,and the mutual-structure term helps to select accurate windows during the structure characteristics of the guidance image are transferred to the output.Experimental results show that MSWGIF reduces the root mean square error(RMSE) by an average of 12.37%,and the average growth rate of correlation(CORR) is 0.07% on average.Additionally,the average growth rate of structure similarity index measure(SSIM) is 0.34%.展开更多
Objective:Early predicting response before neoadjuvant chemotherapy(NAC)is crucial for personalized treatment plans for locally advanced breast cancer patients.We aim to develop a multi-task model using multiscale who...Objective:Early predicting response before neoadjuvant chemotherapy(NAC)is crucial for personalized treatment plans for locally advanced breast cancer patients.We aim to develop a multi-task model using multiscale whole slide images(WSIs)features to predict the response to breast cancer NAC more finely.Methods:This work collected 1,670 whole slide images for training and validation sets,internal testing sets,external testing sets,and prospective testing sets of the weakly-supervised deep learning-based multi-task model(DLMM)in predicting treatment response and pCR to NAC.Our approach models two-by-two feature interactions across scales by employing concatenate fusion of single-scale feature representations,and controls the expressiveness of each representation via a gating-based attention mechanism.Results:In the retrospective analysis,DLMM exhibited excellent predictive performance for the prediction of treatment response,with area under the receiver operating characteristic curves(AUCs)of 0.869[95%confidence interval(95%CI):0.806−0.933]in the internal testing set and 0.841(95%CI:0.814−0.867)in the external testing sets.For the pCR prediction task,DLMM reached AUCs of 0.865(95%CI:0.763−0.964)in the internal testing and 0.821(95%CI:0.763−0.878)in the pooled external testing set.In the prospective testing study,DLMM also demonstrated favorable predictive performance,with AUCs of 0.829(95%CI:0.754−0.903)and 0.821(95%CI:0.692−0.949)in treatment response and pCR prediction,respectively.DLMM significantly outperformed the baseline models in all testing sets(P<0.05).Heatmaps were employed to interpret the decision-making basis of the model.Furthermore,it was discovered that high DLMM scores were associated with immune-related pathways and cells in the microenvironment during biological basis exploration.Conclusions:The DLMM represents a valuable tool that aids clinicians in selecting personalized treatment strategies for breast cancer patients.展开更多
Gamma-ray imaging systems are powerful tools in radiographic diagnosis.However,the recorded images suffer from degradations such as noise,blurring,and downsampling,consequently failing to meet high-precision diagnosti...Gamma-ray imaging systems are powerful tools in radiographic diagnosis.However,the recorded images suffer from degradations such as noise,blurring,and downsampling,consequently failing to meet high-precision diagnostic requirements.In this paper,we propose a novel single-image super-resolution algorithm to enhance the spatial resolution of gamma-ray imaging systems.A mathematical model of the gamma-ray imaging system is established based on maximum a posteriori estimation.Within the plug-and-play framework,the half-quadratic splitting method is employed to decouple the data fidelit term and the regularization term.An image denoiser using convolutional neural networks is adopted as an implicit image prior,referred to as a deep denoiser prior,eliminating the need to explicitly design a regularization term.Furthermore,the impact of the image boundary condition on reconstruction results is considered,and a method for estimating image boundaries is introduced.The results show that the proposed algorithm can effectively addresses boundary artifacts.By increasing the pixel number of the reconstructed images,the proposed algorithm is capable of recovering more details.Notably,in both simulation and real experiments,the proposed algorithm is demonstrated to achieve subpixel resolution,surpassing the Nyquist sampling limit determined by the camera pixel size.展开更多
Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global featu...Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.展开更多
Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic ...Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic imaging techniques,such as bioluminescence tomography(BLT).Nevertheless,nearly every existing DL-based method utilizes an explicit neural representation for the reconstruction problem,which either consumes much memory space or requires various complicated computations.In this paper,we present a neural field(NF)-based image reconstruction scheme for BLT that uses an implicit neural representation.The proposed NFbased method establishes a transformation between the coordinate of an arbitrary spatial point and the source value of the point with a relatively light-weight multilayer perceptron,which has remarkable computational efficiency.Another simple neural network composed of two fully connected layers and a 1D convolutional layer is used to generate the neural features.Results of simulations and experiments show that the proposed NF-based method has similar performance to the photon density complement network and the two-stage network,while consuming fewer floating point operations with fewer model parameters.展开更多
The instance segmentation of impacted teeth in the oral panoramic X-ray images is hotly researched.However,due to the complex structure,low contrast,and complex background of teeth in panoramic X-ray images,the task o...The instance segmentation of impacted teeth in the oral panoramic X-ray images is hotly researched.However,due to the complex structure,low contrast,and complex background of teeth in panoramic X-ray images,the task of instance segmentation is technically tricky.In this study,the contrast between impacted Teeth and periodontal tissues such as gingiva,periodontalmembrane,and alveolar bone is low,resulting in fuzzy boundaries of impacted teeth.Amodel based on Teeth YOLACT is proposed to provide amore efficient and accurate solution for the segmentation of impacted teeth in oral panoramic X-ray films.Firstly,a Multi-scale Res-Transformer Module(MRTM)is designed.In the module,depthwise separable convolutions with different receptive fields are used to enhance the sensitivity of the model to lesion size.Additionally,the Vision Transformer is integrated to improve the model’s ability to perceive global features.Secondly,the Context Interaction-awareness Module(CIaM)is designed to fuse deep and shallow features.The deep semantic features guide the shallow spatial features.Then,the shallow spatial features are embedded into the deep semantic features,and the cross-weighted attention mechanism is used to aggregate the deep and shallow features efficiently,and richer context information is obtained.Thirdly,the Edge-preserving perceptionModule(E2PM)is designed to enhance the teeth edge features.The first-order differential operator is used to get the tooth edge weight,and the perception ability of tooth edge features is improved.The shallow spatial feature is fused by linear mapping,weight concatenation,and matrix multiplication operations to preserve the tooth edge information.Finally,comparison experiments and ablation experiments are conducted on the oral panoramic X-ray image datasets.The results show that the APdet,APseg,ARdet,ARseg,mAPdet,and mAPseg indicators of the proposed model are 89.9%,91.9%,77.4%,77.6%,72.8%,and 73.5%,respectively.This study further verifies the application potential of the method combining multi-scale feature extraction,multi-scale feature fusion,and edge perception enhancement in medical image segmentation,which provides a valuable reference for future related research.展开更多
Photoacoustic computed tomography(PACT)is an innovative biomedical imaging technique that has gained significant application in the field of biomedicine due to its ability to visualize optical contrast with high resol...Photoacoustic computed tomography(PACT)is an innovative biomedical imaging technique that has gained significant application in the field of biomedicine due to its ability to visualize optical contrast with high resolution and deep tissue penetration.However,the inherent challenges associated with photoacoustic signal excitation,propagation and detection often result in suboptimal image quality.To overcome these limitations,researchers have developed various advanced algorithms that span the entire image reconstruction pipeline.This review paper aims to present a detailed analysis of the latest advancements in PACT algorithms and synthesize these algorithms into a coherent framework.We provide tripartite analysis—from signal processing to reconstruction solution to image processing,covering a spectrum of techniques.The principles and methodologies,as well as their applicability and limitations,are thoroughly discussed.The primary objective of this study is to provide a thorough review of advanced algorithms applicable to PACT,offering both theoretical foundations and practical guidance for enhancing the imaging effect of PACT.展开更多
Recently,Mueller matrix(MM)polarimetric imaging-assisted pathology detection methods are showing great potential in clinical diagnosis.However,since our human eyes cannot observe polarized light directly,it raises a n...Recently,Mueller matrix(MM)polarimetric imaging-assisted pathology detection methods are showing great potential in clinical diagnosis.However,since our human eyes cannot observe polarized light directly,it raises a notable challenge for interpreting the measurement results by pathologists who have limited familiarity with polarization images.One feasible approach is to combine MM polarimetric imaging with virtual staining techniques to generate standardized stained images,inheriting the advantages of information-abundant MM polarimetric imaging.In this study,we develop a model using unpaired MM polarimetric images and bright-field images for generating standard hematoxylin and eosin(H&E)stained tissue images.Compared with the existing polarization virtual staining techniques primarily based on the model training with paired images,the proposed Cycle-Consistent Generative Adversarial Networks(CycleGAN)-based model simplifies data acquisition and data preprocessing to a great extent.The outcomes demonstrate the feasibility of training CycleGAN with unpaired polarization images and their corresponding bright-field images as a viable approach,which provides an intuitive manner for pathologists for future polarization-assisted digital pathology.展开更多
Due to hardware limitations,existing hyperspectral(HS)camera often suffer from low spatial/temporal resolution.Recently,it has been prevalent to super-resolve a low reso-lution(LR)HS image into a high resolution(HR)HS...Due to hardware limitations,existing hyperspectral(HS)camera often suffer from low spatial/temporal resolution.Recently,it has been prevalent to super-resolve a low reso-lution(LR)HS image into a high resolution(HR)HS image with a HR RGB(or mul-tispectral)image guidance.Previous approaches for this guided super-resolution task often model the intrinsic characteristic of the desired HR HS image using hand-crafted priors.Recently,researchers pay more attention to deep learning methods with direct supervised or unsupervised learning,which exploit deep prior only from training dataset or testing data.In this article,an efficient convolutional neural network-based method is presented to progressively super-resolve HS image with RGB image guidance.Specif-ically,a progressive HS image super-resolution network is proposed,which progressively super-resolve the LR HS image with pixel shuffled HR RGB image guidance.Then,the super-resolution network is progressively trained with supervised pre-training and un-supervised adaption,where supervised pre-training learns the general prior on training data and unsupervised adaptation generalises the general prior to specific prior for variant testing scenes.The proposed method can effectively exploit prior from training dataset and testing HS and RGB images with spectral-spatial constraint.It has a good general-isation capability,especially for blind HS image super-resolution.Comprehensive experimental results show that the proposed deep progressive learning method out-performs the existing state-of-the-art methods for HS image super-resolution in non-blind and blind cases.展开更多
Due to the selective absorption of light and the existence of a large number of floating media in sea water, underwater images often suffer from color casts and detail blurs. It is therefore necessary to perform color...Due to the selective absorption of light and the existence of a large number of floating media in sea water, underwater images often suffer from color casts and detail blurs. It is therefore necessary to perform color correction and detail restoration. However,the existing enhancement algorithms cannot achieve the desired results. In order to solve the above problems, this paper proposes a multi-stream feature fusion network. First, an underwater image is preprocessed to obtain potential information from the illumination stream, color stream and structure stream by histogram equalization with contrast limitation, gamma correction and white balance, respectively. Next, these three streams and the original raw stream are sent to the residual blocks to extract the features. The features will be subsequently fused. It can enhance feature representation in underwater images. In the meantime, a composite loss function including three terms is used to ensure the quality of the enhanced image from the three aspects of color balance, structure preservation and image smoothness. Therefore, the enhanced image is more in line with human visual perception.Finally, the effectiveness of the proposed method is verified by comparison experiments with many stateof-the-art underwater image enhancement algorithms. Experimental results show that the proposed method provides superior results over them in terms of MSE,PSNR, SSIM, UIQM and UCIQE, and the enhanced images are more similar to their ground truth images.展开更多
Throughout the SMILE mission the satellite will be bombarded by radiation which gradually damages the focal plane devices and degrades their performance.In order to understand the changes of the CCD370s within the sof...Throughout the SMILE mission the satellite will be bombarded by radiation which gradually damages the focal plane devices and degrades their performance.In order to understand the changes of the CCD370s within the soft X-ray Imager,an initial characterisation of the devices has been carried out to give a baseline performance level.Three CCDs have been characterised,the two flight devices and the flight spa re.This has been carried out at the Open University in a bespo ke cleanroom measure ment facility.The results show that there is a cluster of bright pixels in the flight spa re which increases in size with tempe rature.However at the nominal ope rating tempe rature(-120℃) it is within the procure ment specifications.Overall,the devices meet the specifications when ope rating at -120℃ in 6 × 6 binned frame transfer science mode.The se rial charge transfer inefficiency degrades with temperature in full frame mode.However any charge losses are recovered when binning/frame transfer is implemented.展开更多
Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditiona...Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.展开更多
Nano-computed tomography(Nano-CT)is an emerging,high-resolution imaging technique.However,due to their low-light properties,tabletop Nano-CT has to be scanned under long exposure conditions,which the scanning process ...Nano-computed tomography(Nano-CT)is an emerging,high-resolution imaging technique.However,due to their low-light properties,tabletop Nano-CT has to be scanned under long exposure conditions,which the scanning process is time-consuming.For 3D reconstruction data,this paper proposed a lightweight 3D noise reduction method for desktop-level Nano-CT called AAD-ResNet(Axial Attention DeNoise ResNet).The network is framed by theU-net structure.The encoder and decoder are incorporated with the proposed 3D axial attention mechanism and residual dense block.Each layer of the residual dense block can directly access the features of the previous layer,which reduces the redundancy of parameters and improves the efficiency of network training.The 3D axial attention mechanism enhances the correlation between 3D information in the training process and captures the long-distance dependence.It can improve the noise reduction effect and avoid the loss of image structure details.Experimental results show that the network can effectively improve the image quality of a 0.1-s exposure scan to a level close to a 3-s exposure,significantly shortening the sample scanning time.展开更多
Diagnosing various diseases such as glaucoma,age-related macular degeneration,cardiovascular conditions,and diabetic retinopathy involves segmenting retinal blood vessels.The task is particularly challenging when deal...Diagnosing various diseases such as glaucoma,age-related macular degeneration,cardiovascular conditions,and diabetic retinopathy involves segmenting retinal blood vessels.The task is particularly challenging when dealing with color fundus images due to issues like non-uniformillumination,low contrast,and variations in vessel appearance,especially in the presence of different pathologies.Furthermore,the speed of the retinal vessel segmentation system is of utmost importance.With the surge of now available big data,the speed of the algorithm becomes increasingly important,carrying almost equivalent weightage to the accuracy of the algorithm.To address these challenges,we present a novel approach for retinal vessel segmentation,leveraging efficient and robust techniques based on multiscale line detection and mathematical morphology.Our algorithm’s performance is evaluated on two publicly available datasets,namely the Digital Retinal Images for Vessel Extraction dataset(DRIVE)and the Structure Analysis of Retina(STARE)dataset.The experimental results demonstrate the effectiveness of our method,withmean accuracy values of 0.9467 forDRIVE and 0.9535 for STARE datasets,aswell as sensitivity values of 0.6952 forDRIVE and 0.6809 for STARE datasets.Notably,our algorithmexhibits competitive performance with state-of-the-art methods.Importantly,it operates at an average speed of 3.73 s per image for DRIVE and 3.75 s for STARE datasets.It is worth noting that these results were achieved using Matlab scripts containing multiple loops.This suggests that the processing time can be further reduced by replacing loops with vectorization.Thus the proposed algorithm can be deployed in real time applications.In summary,our proposed system strikes a fine balance between swift computation and accuracy that is on par with the best available methods in the field.展开更多
Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial ne...Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.展开更多
In order to solve the problems of artifacts and noise in low-dose computed tomography(CT)images in clinical medical diagnosis,an improved image denoising algorithm under the architecture of generative adversarial netw...In order to solve the problems of artifacts and noise in low-dose computed tomography(CT)images in clinical medical diagnosis,an improved image denoising algorithm under the architecture of generative adversarial network(GAN)was proposed.First,a noise model based on style GAN2 was constructed to estimate the real noise distribution,and the noise information similar to the real noise distribution was generated as the experimental noise data set.Then,a network model with encoder-decoder architecture as the core based on GAN idea was constructed,and the network model was trained with the generated noise data set until it reached the optimal value.Finally,the noise and artifacts in low-dose CT images could be removed by inputting low-dose CT images into the denoising network.The experimental results showed that the constructed network model based on GAN architecture improved the utilization rate of noise feature information and the stability of network training,removed image noise and artifacts,and reconstructed image with rich texture and realistic visual effect.展开更多
基金supported by the Natural Science Foundation of China(62473105,62172118)Nature Science Key Foundation of Guangxi(2021GXNSFDA196002)+1 种基金in part by the Guangxi Key Laboratory of Image and Graphic Intelligent Processing under Grants(GIIP2302,GIIP2303,GIIP2304)Innovation Project of Guang Xi Graduate Education(2024YCXB09,2024YCXS039).
文摘Existing Transformer-based image captioning models typically rely on the self-attention mechanism to capture long-range dependencies,which effectively extracts and leverages the global correlation of image features.However,these models still face challenges in effectively capturing local associations.Moreover,since the encoder extracts global and local association features that focus on different semantic information,semantic noise may occur during the decoding stage.To address these issues,we propose the Local Relationship Enhanced Gated Transformer(LREGT).In the encoder part,we introduce the Local Relationship Enhanced Encoder(LREE),whose core component is the Local Relationship Enhanced Module(LREM).LREM consists of two novel designs:the Local Correlation Perception Module(LCPM)and the Local-Global Fusion Module(LGFM),which are beneficial for generating a comprehensive feature representation that integrates both global and local information.In the decoder part,we propose the Dual-level Multi-branch Gated Decoder(DMGD).It first creates multiple decoding branches to generate multi-perspective contextual feature representations.Subsequently,it employs the Dual-Level Gating Mechanism(DLGM)to model the multi-level relationships of these multi-perspective contextual features,enhancing their fine-grained semantics and intrinsic relationship representations.This ultimately leads to the generation of high-quality and semantically rich image captions.Experiments on the standard MSCOCO dataset demonstrate that LREGT achieves state-of-the-art performance,with a CIDEr score of 140.8 and BLEU-4 score of 41.3,significantly outperforming existing mainstream methods.These results highlight LREGT’s superiority in capturing complex visual relationships and resolving semantic noise during decoding.
基金supported by the Natural Science Foundation of the Anhui Higher Education Institutions of China(Grant Nos.2023AH040149 and 2024AH051915)the Anhui Provincial Natural Science Foundation(Grant No.2208085MF168)+1 种基金the Science and Technology Innovation Tackle Plan Project of Maanshan(Grant No.2024RGZN001)the Scientific Research Fund Project of Anhui Medical University(Grant No.2023xkj122).
文摘Convolutional neural networks(CNNs)-based medical image segmentation technologies have been widely used in medical image segmentation because of their strong representation and generalization abilities.However,due to the inability to effectively capture global information from images,CNNs can easily lead to loss of contours and textures in segmentation results.Notice that the transformer model can effectively capture the properties of long-range dependencies in the image,and furthermore,combining the CNN and the transformer can effectively extract local details and global contextual features of the image.Motivated by this,we propose a multi-branch and multi-scale attention network(M2ANet)for medical image segmentation,whose architecture consists of three components.Specifically,in the first component,we construct an adaptive multi-branch patch module for parallel extraction of image features to reduce information loss caused by downsampling.In the second component,we apply residual block to the well-known convolutional block attention module to enhance the network’s ability to recognize important features of images and alleviate the phenomenon of gradient vanishing.In the third component,we design a multi-scale feature fusion module,in which we adopt adaptive average pooling and position encoding to enhance contextual features,and then multi-head attention is introduced to further enrich feature representation.Finally,we validate the effectiveness and feasibility of the proposed M2ANet method through comparative experiments on four benchmark medical image segmentation datasets,particularly in the context of preserving contours and textures.
基金supported by the First Affiliated Hospital of Xi’an Jiaotong University Teaching Reform Project(Grant No.JG2023-0206 and JG2022-0324).
文摘Objective:In the Radiology Department of Mzuzu Central Hospital(MCH),daily training for radiographers now includes content on Computed Tomography(CT)image quality control and equipment maintenance to ensure the normal,continuous,and stable operation of the 16-slice spiral CT scanner.Methods:Through comprehensive analysis of relevant equipment,we have identified key parameters that significantly impact CT image quality.Innovative optimization strategies and solutions targeting these parameters have been developed and integrated into daily training programs.Furthermore,starting from an examination of prevalent failure modes observed in CT equipment,we delve into essential maintenance and preservation techniques that CT technologists must master to ensure optimal system performance.Results:(1)Crucial factors affecting CT image quality include artifacts,noise,partial volume effects,and surrounding gap phenomena,alongside spatial and density resolutions,CT dose,reconstruction algorithms,and human factors during the scanning process.In the daily training for radiographers,emphasis is placed on strictly implementing image quality control measures at every stage of the CT scanning process and skillfully applying advanced scanning and image processing techniques.By doing so,we can provide clinicians with accurate and reliable imaging references for diagnosis and treatment.(2)Strategies for CT equipment maintenance:①Environmental inspection of the CT room to ensure cleanliness and hygiene.②Rational and accurate operation,including calibration software proficiency.③Regular maintenance and servicing for minimizing machine downtime.④Maintenance of the CT X-ray tube.CT technicians can become proficient in equipment maintenance and upkeep techniques through training,which can significantly extend the service life of CT systems and reduce the occurrence of malfunctions.Conclusion:Through the regular implementation of rigorous CT image quality control training for radiology technicians,coupled with diligent and proactive CT equipment maintenance,we have observed profound and beneficial impacts on improving image quality.The accuracy and fidelity of radiological data ultimately leads to more accurate diagnoses and effective treatments.
基金supported by the National Key R&D Program of China(No.2022YFB3205101)NSAF(No.U2230116)。
文摘To improve image quality under low illumination conditions,a novel low-light image enhancement method is proposed in this paper based on multi-illumination estimation and multi-scale fusion(MIMS).Firstly,the illumination is processed by contrast-limited adaptive histogram equalization(CLAHE),adaptive complementary gamma function(ACG),and adaptive detail preserving S-curve(ADPS),respectively,to obtain three components.Then,the fusion-relevant features,exposure,and color contrast are selected as the weight maps.Subsequently,these components and weight maps are fused through multi-scale to generate enhanced illumination.Finally,the enhanced images are obtained by multiplying the enhanced illumination and reflectance.Compared with existing approaches,this proposed method achieves an average increase of 0.81%and 2.89%in the structural similarity index measurement(SSIM)and peak signal-to-noise ratio(PSNR),and a decrease of 6.17%and 32.61%in the natural image quality evaluator(NIQE)and gradient magnitude similarity deviation(GMSD),respectively.
基金supported by the National Key Research and Development Program of China (No.2019YFB2204302)。
文摘Although guided image filtering(GIF) is known for preserving edges and fast computation,it may produce inaccurate outputs in depth map restoration.In this paper,a novel confidence-weighted GIF called mutual-structure weighted GIF(MSWGIF) is proposed,which replaces the mean filtering strategy in GIF during handling overlapping windows.The confidence value is composed of a depth term and a mutual-structure term,where the depth term is utilized to protect the edges of the output,and the mutual-structure term helps to select accurate windows during the structure characteristics of the guidance image are transferred to the output.Experimental results show that MSWGIF reduces the root mean square error(RMSE) by an average of 12.37%,and the average growth rate of correlation(CORR) is 0.07% on average.Additionally,the average growth rate of structure similarity index measure(SSIM) is 0.34%.
基金supported by the National Natural Science Foundation of China(No.82371933)the National Natural Science Foundation of Shandong Province of China(No.ZR2021MH120)+1 种基金the Taishan Scholars Project(No.tsqn202211378)the Shandong Provincial Natural Science Foundation for Excellent Young Scholars(No.ZR2024YQ075).
文摘Objective:Early predicting response before neoadjuvant chemotherapy(NAC)is crucial for personalized treatment plans for locally advanced breast cancer patients.We aim to develop a multi-task model using multiscale whole slide images(WSIs)features to predict the response to breast cancer NAC more finely.Methods:This work collected 1,670 whole slide images for training and validation sets,internal testing sets,external testing sets,and prospective testing sets of the weakly-supervised deep learning-based multi-task model(DLMM)in predicting treatment response and pCR to NAC.Our approach models two-by-two feature interactions across scales by employing concatenate fusion of single-scale feature representations,and controls the expressiveness of each representation via a gating-based attention mechanism.Results:In the retrospective analysis,DLMM exhibited excellent predictive performance for the prediction of treatment response,with area under the receiver operating characteristic curves(AUCs)of 0.869[95%confidence interval(95%CI):0.806−0.933]in the internal testing set and 0.841(95%CI:0.814−0.867)in the external testing sets.For the pCR prediction task,DLMM reached AUCs of 0.865(95%CI:0.763−0.964)in the internal testing and 0.821(95%CI:0.763−0.878)in the pooled external testing set.In the prospective testing study,DLMM also demonstrated favorable predictive performance,with AUCs of 0.829(95%CI:0.754−0.903)and 0.821(95%CI:0.692−0.949)in treatment response and pCR prediction,respectively.DLMM significantly outperformed the baseline models in all testing sets(P<0.05).Heatmaps were employed to interpret the decision-making basis of the model.Furthermore,it was discovered that high DLMM scores were associated with immune-related pathways and cells in the microenvironment during biological basis exploration.Conclusions:The DLMM represents a valuable tool that aids clinicians in selecting personalized treatment strategies for breast cancer patients.
基金supported by the National Natural Science Foundation of China(Grant No.12175183)。
文摘Gamma-ray imaging systems are powerful tools in radiographic diagnosis.However,the recorded images suffer from degradations such as noise,blurring,and downsampling,consequently failing to meet high-precision diagnostic requirements.In this paper,we propose a novel single-image super-resolution algorithm to enhance the spatial resolution of gamma-ray imaging systems.A mathematical model of the gamma-ray imaging system is established based on maximum a posteriori estimation.Within the plug-and-play framework,the half-quadratic splitting method is employed to decouple the data fidelit term and the regularization term.An image denoiser using convolutional neural networks is adopted as an implicit image prior,referred to as a deep denoiser prior,eliminating the need to explicitly design a regularization term.Furthermore,the impact of the image boundary condition on reconstruction results is considered,and a method for estimating image boundaries is introduced.The results show that the proposed algorithm can effectively addresses boundary artifacts.By increasing the pixel number of the reconstructed images,the proposed algorithm is capable of recovering more details.Notably,in both simulation and real experiments,the proposed algorithm is demonstrated to achieve subpixel resolution,surpassing the Nyquist sampling limit determined by the camera pixel size.
基金National Key Research and Development Program of China,Grant/Award Number:2018YFE0206900China Postdoctoral Science Foundation,Grant/Award Number:2023M731204+2 种基金The Open Project of Key Laboratory for Quality Evaluation of Ultrasound Surgical Equipment of National Medical Products Administration,Grant/Award Number:SMDTKL-2023-1-01The Hubei Province Key Research and Development Project,Grant/Award Number:2023BCB007CAAI-Huawei MindSpore Open Fund。
文摘Convolutional neural network(CNN)with the encoder-decoder structure is popular in medical image segmentation due to its excellent local feature extraction ability but it faces limitations in capturing the global feature.The transformer can extract the global information well but adapting it to small medical datasets is challenging and its computational complexity can be heavy.In this work,a serial and parallel network is proposed for the accurate 3D medical image segmentation by combining CNN and transformer and promoting feature interactions across various semantic levels.The core components of the proposed method include the cross window self-attention based transformer(CWST)and multi-scale local enhanced(MLE)modules.The CWST module enhances the global context understanding by partitioning 3D images into non-overlapping windows and calculating sparse global attention between windows.The MLE module selectively fuses features by computing the voxel attention between different branch features,and uses convolution to strengthen the dense local information.The experiments on the prostate,atrium,and pancreas MR/CT image datasets consistently demonstrate the advantage of the proposed method over six popular segmentation models in both qualitative evaluation and quantitative indexes such as dice similarity coefficient,Intersection over Union,95%Hausdorff distance and average symmetric surface distance.
基金supported in part by the National Natural Science Foundation of China(62101278,62001379,62271023)Beijing Natural Science Foundation(7242269).
文摘Deep learning(DL)-based image reconstruction methods have garnered increasing interest in the last few years.Numerous studies demonstrate that DL-based reconstruction methods function admirably in optical tomographic imaging techniques,such as bioluminescence tomography(BLT).Nevertheless,nearly every existing DL-based method utilizes an explicit neural representation for the reconstruction problem,which either consumes much memory space or requires various complicated computations.In this paper,we present a neural field(NF)-based image reconstruction scheme for BLT that uses an implicit neural representation.The proposed NFbased method establishes a transformation between the coordinate of an arbitrary spatial point and the source value of the point with a relatively light-weight multilayer perceptron,which has remarkable computational efficiency.Another simple neural network composed of two fully connected layers and a 1D convolutional layer is used to generate the neural features.Results of simulations and experiments show that the proposed NF-based method has similar performance to the photon density complement network and the two-stage network,while consuming fewer floating point operations with fewer model parameters.
基金supported in part by the National Natural Science Foundation of China(Grant No.62062003)Natural Science Foundation of Ningxia(Grant No.2023AAC03293).
文摘The instance segmentation of impacted teeth in the oral panoramic X-ray images is hotly researched.However,due to the complex structure,low contrast,and complex background of teeth in panoramic X-ray images,the task of instance segmentation is technically tricky.In this study,the contrast between impacted Teeth and periodontal tissues such as gingiva,periodontalmembrane,and alveolar bone is low,resulting in fuzzy boundaries of impacted teeth.Amodel based on Teeth YOLACT is proposed to provide amore efficient and accurate solution for the segmentation of impacted teeth in oral panoramic X-ray films.Firstly,a Multi-scale Res-Transformer Module(MRTM)is designed.In the module,depthwise separable convolutions with different receptive fields are used to enhance the sensitivity of the model to lesion size.Additionally,the Vision Transformer is integrated to improve the model’s ability to perceive global features.Secondly,the Context Interaction-awareness Module(CIaM)is designed to fuse deep and shallow features.The deep semantic features guide the shallow spatial features.Then,the shallow spatial features are embedded into the deep semantic features,and the cross-weighted attention mechanism is used to aggregate the deep and shallow features efficiently,and richer context information is obtained.Thirdly,the Edge-preserving perceptionModule(E2PM)is designed to enhance the teeth edge features.The first-order differential operator is used to get the tooth edge weight,and the perception ability of tooth edge features is improved.The shallow spatial feature is fused by linear mapping,weight concatenation,and matrix multiplication operations to preserve the tooth edge information.Finally,comparison experiments and ablation experiments are conducted on the oral panoramic X-ray image datasets.The results show that the APdet,APseg,ARdet,ARseg,mAPdet,and mAPseg indicators of the proposed model are 89.9%,91.9%,77.4%,77.6%,72.8%,and 73.5%,respectively.This study further verifies the application potential of the method combining multi-scale feature extraction,multi-scale feature fusion,and edge perception enhancement in medical image segmentation,which provides a valuable reference for future related research.
基金supported by Beijing Natural Science Foundation(7232146)National Natural Science Foundation of China(NSFC)Grant(62475277,62105355,82122034,82327805,81927807,62275062)+4 种基金Strategic Priority Research Program of the Chinese Academy of Sciences(XDB0930000)Shenzhen Science and Technology Innovation Grant(JCYJ20220531100409023,JCYJ20210324101403010,JCYJ20220818101403008)Project of Shandong Innovation and Startup Community of High-end Medical Apparatus and Instruments Grant(2021-SGTTXM005)Shandong Province Technology Innovation Guidance Plan(Central Leading Local Science and Technology Development Fund,YDZX2023115)Taishan Scholar Special Funding Project of Shandong Province.
文摘Photoacoustic computed tomography(PACT)is an innovative biomedical imaging technique that has gained significant application in the field of biomedicine due to its ability to visualize optical contrast with high resolution and deep tissue penetration.However,the inherent challenges associated with photoacoustic signal excitation,propagation and detection often result in suboptimal image quality.To overcome these limitations,researchers have developed various advanced algorithms that span the entire image reconstruction pipeline.This review paper aims to present a detailed analysis of the latest advancements in PACT algorithms and synthesize these algorithms into a coherent framework.We provide tripartite analysis—from signal processing to reconstruction solution to image processing,covering a spectrum of techniques.The principles and methodologies,as well as their applicability and limitations,are thoroughly discussed.The primary objective of this study is to provide a thorough review of advanced algorithms applicable to PACT,offering both theoretical foundations and practical guidance for enhancing the imaging effect of PACT.
基金Shenzhen Key Fundamental Research Project(No.JCYJ20210324120012035).
文摘Recently,Mueller matrix(MM)polarimetric imaging-assisted pathology detection methods are showing great potential in clinical diagnosis.However,since our human eyes cannot observe polarized light directly,it raises a notable challenge for interpreting the measurement results by pathologists who have limited familiarity with polarization images.One feasible approach is to combine MM polarimetric imaging with virtual staining techniques to generate standardized stained images,inheriting the advantages of information-abundant MM polarimetric imaging.In this study,we develop a model using unpaired MM polarimetric images and bright-field images for generating standard hematoxylin and eosin(H&E)stained tissue images.Compared with the existing polarization virtual staining techniques primarily based on the model training with paired images,the proposed Cycle-Consistent Generative Adversarial Networks(CycleGAN)-based model simplifies data acquisition and data preprocessing to a great extent.The outcomes demonstrate the feasibility of training CycleGAN with unpaired polarization images and their corresponding bright-field images as a viable approach,which provides an intuitive manner for pathologists for future polarization-assisted digital pathology.
基金National Key R&D Program of China,Grant/Award Number:2022YFC3300704National Natural Science Foundation of China,Grant/Award Numbers:62171038,62088101,62006023。
文摘Due to hardware limitations,existing hyperspectral(HS)camera often suffer from low spatial/temporal resolution.Recently,it has been prevalent to super-resolve a low reso-lution(LR)HS image into a high resolution(HR)HS image with a HR RGB(or mul-tispectral)image guidance.Previous approaches for this guided super-resolution task often model the intrinsic characteristic of the desired HR HS image using hand-crafted priors.Recently,researchers pay more attention to deep learning methods with direct supervised or unsupervised learning,which exploit deep prior only from training dataset or testing data.In this article,an efficient convolutional neural network-based method is presented to progressively super-resolve HS image with RGB image guidance.Specif-ically,a progressive HS image super-resolution network is proposed,which progressively super-resolve the LR HS image with pixel shuffled HR RGB image guidance.Then,the super-resolution network is progressively trained with supervised pre-training and un-supervised adaption,where supervised pre-training learns the general prior on training data and unsupervised adaptation generalises the general prior to specific prior for variant testing scenes.The proposed method can effectively exploit prior from training dataset and testing HS and RGB images with spectral-spatial constraint.It has a good general-isation capability,especially for blind HS image super-resolution.Comprehensive experimental results show that the proposed deep progressive learning method out-performs the existing state-of-the-art methods for HS image super-resolution in non-blind and blind cases.
基金supported by the national key research and development program (No.2020YFB1806608)Jiangsu natural science foundation for distinguished young scholars (No.BK20220054)。
文摘Due to the selective absorption of light and the existence of a large number of floating media in sea water, underwater images often suffer from color casts and detail blurs. It is therefore necessary to perform color correction and detail restoration. However,the existing enhancement algorithms cannot achieve the desired results. In order to solve the above problems, this paper proposes a multi-stream feature fusion network. First, an underwater image is preprocessed to obtain potential information from the illumination stream, color stream and structure stream by histogram equalization with contrast limitation, gamma correction and white balance, respectively. Next, these three streams and the original raw stream are sent to the residual blocks to extract the features. The features will be subsequently fused. It can enhance feature representation in underwater images. In the meantime, a composite loss function including three terms is used to ensure the quality of the enhanced image from the three aspects of color balance, structure preservation and image smoothness. Therefore, the enhanced image is more in line with human visual perception.Finally, the effectiveness of the proposed method is verified by comparison experiments with many stateof-the-art underwater image enhancement algorithms. Experimental results show that the proposed method provides superior results over them in terms of MSE,PSNR, SSIM, UIQM and UCIQE, and the enhanced images are more similar to their ground truth images.
文摘Throughout the SMILE mission the satellite will be bombarded by radiation which gradually damages the focal plane devices and degrades their performance.In order to understand the changes of the CCD370s within the soft X-ray Imager,an initial characterisation of the devices has been carried out to give a baseline performance level.Three CCDs have been characterised,the two flight devices and the flight spa re.This has been carried out at the Open University in a bespo ke cleanroom measure ment facility.The results show that there is a cluster of bright pixels in the flight spa re which increases in size with tempe rature.However at the nominal ope rating tempe rature(-120℃) it is within the procure ment specifications.Overall,the devices meet the specifications when ope rating at -120℃ in 6 × 6 binned frame transfer science mode.The se rial charge transfer inefficiency degrades with temperature in full frame mode.However any charge losses are recovered when binning/frame transfer is implemented.
基金funded by the National Natural Science Foundation of China(62125504,61827825,and 31901059)Zhejiang Provincial Ten Thousand Plan for Young Top Talents(2020R52001)Open Project Program of Wuhan National Laboratory for Optoelectronics(2021WNLOKF007).
文摘Structured illumination microscopy(SIM)achieves super-resolution(SR)by modulating the high-frequency information of the sample into the passband of the optical system and subsequent image reconstruction.The traditional Wiener-filtering-based reconstruction algorithm operates in the Fourier domain,it requires prior knowledge of the sinusoidal illumination patterns which makes the time-consuming procedure of parameter estimation to raw datasets necessary,besides,the parameter estimation is sensitive to noise or aberration-induced pattern distortion which leads to reconstruction artifacts.Here,we propose a spatial-domain image reconstruction method that does not require parameter estimation but calculates patterns from raw datasets,and a reconstructed image can be obtained just by calculating the spatial covariance of differential calculated patterns and differential filtered datasets(the notch filtering operation is performed to the raw datasets for attenuating and compensating the optical transfer function(OTF)).Experiments on reconstructing raw datasets including nonbiological,biological,and simulated samples demonstrate that our method has SR capability,high reconstruction speed,and high robustness to aberration and noise.
基金supported by the National Natural Science Foundation of China(62201618).
文摘Nano-computed tomography(Nano-CT)is an emerging,high-resolution imaging technique.However,due to their low-light properties,tabletop Nano-CT has to be scanned under long exposure conditions,which the scanning process is time-consuming.For 3D reconstruction data,this paper proposed a lightweight 3D noise reduction method for desktop-level Nano-CT called AAD-ResNet(Axial Attention DeNoise ResNet).The network is framed by theU-net structure.The encoder and decoder are incorporated with the proposed 3D axial attention mechanism and residual dense block.Each layer of the residual dense block can directly access the features of the previous layer,which reduces the redundancy of parameters and improves the efficiency of network training.The 3D axial attention mechanism enhances the correlation between 3D information in the training process and captures the long-distance dependence.It can improve the noise reduction effect and avoid the loss of image structure details.Experimental results show that the network can effectively improve the image quality of a 0.1-s exposure scan to a level close to a 3-s exposure,significantly shortening the sample scanning time.
文摘Diagnosing various diseases such as glaucoma,age-related macular degeneration,cardiovascular conditions,and diabetic retinopathy involves segmenting retinal blood vessels.The task is particularly challenging when dealing with color fundus images due to issues like non-uniformillumination,low contrast,and variations in vessel appearance,especially in the presence of different pathologies.Furthermore,the speed of the retinal vessel segmentation system is of utmost importance.With the surge of now available big data,the speed of the algorithm becomes increasingly important,carrying almost equivalent weightage to the accuracy of the algorithm.To address these challenges,we present a novel approach for retinal vessel segmentation,leveraging efficient and robust techniques based on multiscale line detection and mathematical morphology.Our algorithm’s performance is evaluated on two publicly available datasets,namely the Digital Retinal Images for Vessel Extraction dataset(DRIVE)and the Structure Analysis of Retina(STARE)dataset.The experimental results demonstrate the effectiveness of our method,withmean accuracy values of 0.9467 forDRIVE and 0.9535 for STARE datasets,aswell as sensitivity values of 0.6952 forDRIVE and 0.6809 for STARE datasets.Notably,our algorithmexhibits competitive performance with state-of-the-art methods.Importantly,it operates at an average speed of 3.73 s per image for DRIVE and 3.75 s for STARE datasets.It is worth noting that these results were achieved using Matlab scripts containing multiple loops.This suggests that the processing time can be further reduced by replacing loops with vectorization.Thus the proposed algorithm can be deployed in real time applications.In summary,our proposed system strikes a fine balance between swift computation and accuracy that is on par with the best available methods in the field.
文摘Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue.Convolutional neural network(CNN)and generative adversarial network(GAN)are pivotal inmedical image registration.However,existing methods often struggle with severe interference and deformation,as seen in histological images of conditions like Cushing’s disease.We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator inGAN.In this study,we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration.To begin with,the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks,characterized by implicitly extracting feature descriptors of specific modalities.Additionally,modal feature description layers and registration layers collaborate in unsupervised optimization,facilitating faster convergence and more precise results.Lastly,experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database(MNIST),eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation(CRCS)dataset on the Cushing’s disease.Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency,while also exhibiting robustness across different image types.
基金supported by National Natural Science Foundation of China(No.11802272)China Postdoctoral Science Foundation(No.2019M651085)。
文摘In order to solve the problems of artifacts and noise in low-dose computed tomography(CT)images in clinical medical diagnosis,an improved image denoising algorithm under the architecture of generative adversarial network(GAN)was proposed.First,a noise model based on style GAN2 was constructed to estimate the real noise distribution,and the noise information similar to the real noise distribution was generated as the experimental noise data set.Then,a network model with encoder-decoder architecture as the core based on GAN idea was constructed,and the network model was trained with the generated noise data set until it reached the optimal value.Finally,the noise and artifacts in low-dose CT images could be removed by inputting low-dose CT images into the denoising network.The experimental results showed that the constructed network model based on GAN architecture improved the utilization rate of noise feature information and the stability of network training,removed image noise and artifacts,and reconstructed image with rich texture and realistic visual effect.
