In this paper,we present TexPro,a novel method for high-fidelity material generation for input 3D meshes given text prompts.Unlike existing text-conditioned texture generation methods that typically generate RGB textu...In this paper,we present TexPro,a novel method for high-fidelity material generation for input 3D meshes given text prompts.Unlike existing text-conditioned texture generation methods that typically generate RGB textures with baked lighting,TexPro is able to produce diverse texture maps via procedural material modeling,which enables physically-based rendering,relighting,and additional benefits inherent to procedural materials.Specifically,we first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model.We then derive texture maps through rendering-based optimization with recent differentiable procedural materials.To this end,we design several techniques to handle the misalignment between the generated multi-view images and 3D meshes,and introduce a novel material agent that enhances material classification and matching by exploring both part-level understanding and object-aware material reasoning.Experiments demonstrate the superiority of the proposed method over existing SOTAs,and its capability of relighting.展开更多
基金supported by the National Natural Science Foundation of China(No.62441222)the Information Technology Center and State Key Lab of CAD&CG,Zhejiang University。
文摘In this paper,we present TexPro,a novel method for high-fidelity material generation for input 3D meshes given text prompts.Unlike existing text-conditioned texture generation methods that typically generate RGB textures with baked lighting,TexPro is able to produce diverse texture maps via procedural material modeling,which enables physically-based rendering,relighting,and additional benefits inherent to procedural materials.Specifically,we first generate multi-view reference images given the input textual prompt by employing the latest text-to-image model.We then derive texture maps through rendering-based optimization with recent differentiable procedural materials.To this end,we design several techniques to handle the misalignment between the generated multi-view images and 3D meshes,and introduce a novel material agent that enhances material classification and matching by exploring both part-level understanding and object-aware material reasoning.Experiments demonstrate the superiority of the proposed method over existing SOTAs,and its capability of relighting.
