This work provides a new multimodal fusion generative adversarial net(GAN)model,Multiple Conditions Transform W-net(MCSTransWnet),which primarily uses femtosecond laser arcuate keratotomy surgical parameters and preop...This work provides a new multimodal fusion generative adversarial net(GAN)model,Multiple Conditions Transform W-net(MCSTransWnet),which primarily uses femtosecond laser arcuate keratotomy surgical parameters and preoperative corneal topography to predict postoperative corneal topography in astigmatism-corrected patients.The MCSTransWnet model comprises a generator and a discriminator,and the generator is composed of two sub-generators.The first sub-generator extracts features using the U-net model,vision transform(ViT)and a multi-parameter conditional module branch.The second sub-generator uses a U-net network for further image denoising.The discriminator uses the pixel discriminator in Pix2Pix.Currently,most GAN models are convolutional neural networks;however,due to their feature extraction locality,it is difficult to comprehend the relationships among global features.Thus,we added a vision Transform network as the model branch to extract the global features.It is normally difficult to train the transformer,and image noise and geometric information loss are likely.Hence,we adopted the standard U-net fusion scheme and transform network as the generator,so that global features,local features,and rich image details could be obtained simultaneously.Our experimental results clearly demonstrate that MCSTransWnet successfully predicts postoperative corneal topographies(structural similarity=0.765,peak signal-to-noise ratio=16.012,and Fréchet inception distance=9.264).Using this technique to obtain the rough shape of the postoperative corneal topography in advance gives clinicians more references and guides changes to surgical planning and improves the success rate of surgery.展开更多
基金National Natural Science Foundation of China(Grant numbers 11872262,12172243,and 12072218)Research Funds of Shanxi Transformation and Comprehensive Reform Demonstration Zone(Grant number 2018KJCX04)+7 种基金Fund for Shanxi“1331 Project”and supported by the Fundamental Research Program of Shanxi Province(Grant number 202203021211006)Shenzhen Fund for Guangdong Provincial High-level Clinical Key Specialties(Grant number SZGSP014)Sanming Project of Medicine in Shenzhen(Grant number SZSM202011015)Shenzhen Fundamental Research Program(Grant number JCYJ20220818103207015)Shenzhen Science and Technology Program(Grant number JCYJ20220530153604010)Medical Major Research Projects in Shanxi Province(Grant number 2021XM11)Scientific Innovation Plan of the Universities in Shanxi Province(Grant number 2021L575)Shanxi Scholarship Council of China(Grant number 2020-149).
文摘This work provides a new multimodal fusion generative adversarial net(GAN)model,Multiple Conditions Transform W-net(MCSTransWnet),which primarily uses femtosecond laser arcuate keratotomy surgical parameters and preoperative corneal topography to predict postoperative corneal topography in astigmatism-corrected patients.The MCSTransWnet model comprises a generator and a discriminator,and the generator is composed of two sub-generators.The first sub-generator extracts features using the U-net model,vision transform(ViT)and a multi-parameter conditional module branch.The second sub-generator uses a U-net network for further image denoising.The discriminator uses the pixel discriminator in Pix2Pix.Currently,most GAN models are convolutional neural networks;however,due to their feature extraction locality,it is difficult to comprehend the relationships among global features.Thus,we added a vision Transform network as the model branch to extract the global features.It is normally difficult to train the transformer,and image noise and geometric information loss are likely.Hence,we adopted the standard U-net fusion scheme and transform network as the generator,so that global features,local features,and rich image details could be obtained simultaneously.Our experimental results clearly demonstrate that MCSTransWnet successfully predicts postoperative corneal topographies(structural similarity=0.765,peak signal-to-noise ratio=16.012,and Fréchet inception distance=9.264).Using this technique to obtain the rough shape of the postoperative corneal topography in advance gives clinicians more references and guides changes to surgical planning and improves the success rate of surgery.
