针对深度学习中的生成对抗网络GAN的模式坍塌和梯度消失问题,通过Wasserstein距离和梯度惩罚的方法,缓解模式坍塌和梯度消失。主要方法包括使用Wasserstein距离代替JS散度改进损失函数,同时防止梯度消失;利用梯度惩罚法强制判别器的连...针对深度学习中的生成对抗网络GAN的模式坍塌和梯度消失问题,通过Wasserstein距离和梯度惩罚的方法,缓解模式坍塌和梯度消失。主要方法包括使用Wasserstein距离代替JS散度改进损失函数,同时防止梯度消失;利用梯度惩罚法强制判别器的连续性约束,确保梯度稳定,平衡生成器与判别器的训练速度;采用从简单到复杂的训练策略优化,平滑过渡,防止判别器过强,同时也防止梯度消失。对传统GAN和改进GAN进行了对比,其生成器和判别器的Loss曲线显示改进后较为稳定。Aiming at the problem of pattern collapse and gradient disappearance of generative adversarial network GAN in deep learning, Wasserstein distance and gradient penalty are used to alleviate the pattern collapse and gradient disappearance. The main methods include using Wasserstein distance instead of JS divergence to improve the loss function, while preventing the gradient from disappearing. The gradient penalty method is used to force the continuity constraint of discriminator to ensure the stability of gradient and balance the training speed of generator and discriminator. Smooth transitions are optimized using training strategies from simple to complex, preventing the discriminator from being too strong and also preventing the gradient from disappearing. A comparison is made between the traditional GAN and the improved GAN, the loss curves of its generator and discriminator show that the improved one is more stable.展开更多
文摘针对深度学习中的生成对抗网络GAN的模式坍塌和梯度消失问题,通过Wasserstein距离和梯度惩罚的方法,缓解模式坍塌和梯度消失。主要方法包括使用Wasserstein距离代替JS散度改进损失函数,同时防止梯度消失;利用梯度惩罚法强制判别器的连续性约束,确保梯度稳定,平衡生成器与判别器的训练速度;采用从简单到复杂的训练策略优化,平滑过渡,防止判别器过强,同时也防止梯度消失。对传统GAN和改进GAN进行了对比,其生成器和判别器的Loss曲线显示改进后较为稳定。Aiming at the problem of pattern collapse and gradient disappearance of generative adversarial network GAN in deep learning, Wasserstein distance and gradient penalty are used to alleviate the pattern collapse and gradient disappearance. The main methods include using Wasserstein distance instead of JS divergence to improve the loss function, while preventing the gradient from disappearing. The gradient penalty method is used to force the continuity constraint of discriminator to ensure the stability of gradient and balance the training speed of generator and discriminator. Smooth transitions are optimized using training strategies from simple to complex, preventing the discriminator from being too strong and also preventing the gradient from disappearing. A comparison is made between the traditional GAN and the improved GAN, the loss curves of its generator and discriminator show that the improved one is more stable.
