摘要
由于缺乏带有像素级标注的公开工笔画数据集,使得图像分割技术在工笔画领域的发展严重受阻。工笔画具有物象与背景颜色纹理相似、使用晕染渐变导致物象边界模糊等特性,给图像分割带来了挑战,SAM的出现为解决这些挑战带来新的可能性。尽管SAM在自然图像领域里展现出惊人分割能力和零样本泛化能力,但在处理工笔画图像时存在对物象不敏感、前景背景混淆等问题。针对上述问题,首先建立了一个包含403幅图像的花鸟主题工笔画数据集SegTCRP,其中包含5类前景对象。随后,采用LoRA方法对SAM进行微调,使其适应工笔画图像的特点。此外,提出了一种新的SAM边界框提示增强方法BPA-SAM,通过借助U-Net在边界框提示范围内基于一定策略辅助生成额外点提示来改善SAM前景背景混淆的问题。最终,实验验证了BPA-SAM较原始SAM在边界框提示条件下的分割性能提升了7.1%,为SAM在工笔画领域的图像分割应用奠定了基础。
Due to the lack of publicly available meticulously annotated datasets for traditional Chinese realistic painting,the development of image segmentation techniques in this field is severely hindered.Traditional Chinese realistic painting exhibits characteristics such as similarity in object and background color textures,as well as blurred object boundaries due to the use of gradient transitions,posing challenges for image segmentation.The emergence of the segment anything model(SAM)presents new possibilities for addressing these challenges.Despite SAM demonstrating remarkable segmentation capabilities and zero-shot generalization in the natural image domain,it faces issues of insensitivity to object details and foreground-background confusion when processing traditional Chinese realistic painting.To address these issues,a segmented Traditional Chinese realistic painting dataset themed around flowers and birds was constructed,comprising 403 images with 5 classes of fore-ground objects.Subsequently,we employed the LoRA(Low-Rank Adaptation)method was employed to fine-tune SAM,enabling it to adapt to the characteristics of traditional Chinese realistic paintings.Additionally,a novel boundary box prompting enhancement method called BPA-SAM was proposed,based on the U-Net model,to address fore-ground-background confusion by generating point prompts within the boundary box range.Ultimately,experiments confirmed that our approach improved SAM’s segmentation performance by 7.1%under boundary box prompting conditions,establishing a foundation for SAM’s image segmentation applications in the traditional Chinese realistic painting domain.
作者
张天圣
朱闽峰
任怡雯
王琛涵
张立冬
张玮
陈为
ZHANG Tiansheng;ZHU Minfeng;REN Yiwen;WANG Chenhan;ZHANG Lidong;ZHANG Wei;CHEN Wei(State Key Laboratory of CAD&CG,Zhejiang University,Hangzhou Zhejiang 310058,China;School of Software Technology,Zhejiang University,Hangzhou Zhejiang 310058,China;College of Computer Science and Technology,Zhejiang University,Hangzhou Zhejiang 310058,China;Hangzhou City University,Hangzhou Zhejiang 310015,China)
出处
《图学学报》
北大核心
2025年第2期322-331,共10页
Journal of Graphics
基金
国家自然科学基金(62132017)
浙江省重点研发“尖兵”攻关计划(2023C01119)
浙江省自然科学基金(LD24F020011,Q24F020006)。
关键词
深度学习
图像分割
工笔画
提示增强
计算机视觉
deep learning
image segmentation
traditional Chinese realistic painting
prompt augmentation
computer vision
