Finding suitable initial noise that retains the original image’s information is crucial for image-to-image(I2I)translation using text-to-image(T2I)diffusion models.A common approach is to add random noise directly to...Finding suitable initial noise that retains the original image’s information is crucial for image-to-image(I2I)translation using text-to-image(T2I)diffusion models.A common approach is to add random noise directly to the original image,as in SDEdit.However,we have observed that this can result in“semantic discrepancy”issues,wherein T2I diffusion models misinterpret the semantic relationships and generate content not present in the original image.We identify that the noise introduced by SDEdit disrupts the semantic integrity of the image,leading to unintended associations between unrelated regions after U-Net upsampling.Building on the widely-used latent diffusion model,Stable Diffusion,we propose a training-free,plugand-play method to alleviate semantic discrepancy and enhance the fidelity of the translated image.By leveraging the deterministic nature of denoising diffusion implicit models(DDIMs)inversion,we correct the erroneous features and correlations from the original generative process with accurate ones from DDIM inversion.This approach alleviates semantic discrepancy and surpasses recent DDIM-inversion-based methods such as PnP with fewer priors,achieving a speedup of 11.2 times in experiments conducted on COCO,ImageNet,and ImageNet-R datasets across multiple I2I translation tasks.展开更多
The glinty details from complex microstructures significantly enhance rendering realism.However,the previous methods use high-resolution normal maps to define each micro-geometry,which requires huge memory overhead.Th...The glinty details from complex microstructures significantly enhance rendering realism.However,the previous methods use high-resolution normal maps to define each micro-geometry,which requires huge memory overhead.This paper observes that many self-similarity materials have independent structural characteristics,which we define as tiny example microstructures.We propose a procedural model to represent microstructures implicitly by performing spatial transformations and spatial distribution on tiny examples.Furthermore,we precompute normal distribution functions(NDFs)by 4D Gaussians for tiny examples and store them in multi-scale NDF maps.Combined with a tiny example based NDF evaluation method,complex glinty surfaces can be rendered simply by texture sampling.The experimental results show that our tiny example based the microstructure rendering method is GPU-friendly,successfully reproducing high-frequency reflection features of different microstructures in real time with low memory and computational overhead.展开更多
Deformation is an important research topic in graphics.There are two key issues in mesh deformation:(1) selfintersection and(2) volume preserving.In this paper,we present a new method to construct a vector field for v...Deformation is an important research topic in graphics.There are two key issues in mesh deformation:(1) selfintersection and(2) volume preserving.In this paper,we present a new method to construct a vector field for volume-preserving mesh deformation of free-form objects.Volume-preserving is an inherent feature of a curl vector field.Since the field lines of the curl vector field will never intersect with each other,a mesh deformed under a curl vector field can avoid self-intersection between field lines.Designing the vector field based on curl is useful in preserving graphic features and preventing self-intersection.Our proposed algorithm introduces distance field into vector field construction;as a result,the shape of the curl vector field is closely related to the object shape.We define the construction of the curl vector field for translation and rotation and provide some special effects such as twisting and bending.Taking into account the information of the object,this approach can provide easy and intuitive construction for free-form objects.Experimental results show that the approach works effectively in real-time animation.展开更多
Currently,most existing inductive relation prediction approaches are based on subgraph structures,with subgraph features extracted using graph neural networks to predict relations.However,subgraphs may contain disconn...Currently,most existing inductive relation prediction approaches are based on subgraph structures,with subgraph features extracted using graph neural networks to predict relations.However,subgraphs may contain disconnected regions,which usually represent different semantic ranges.Because not all semantic information about the regions is helpful in relation prediction,we propose a relation prediction model based on a disentangled subgraph structure and implement a feature updating approach based on relevant semantic aggregation.To indirectly achieve the disentangled subgraph structure from a semantic perspective,the mapping of entity features into different semantic spaces and the aggregation of related semantics on each semantic space are updated.The disentangled model can focus on features having higher semantic relevance in the prediction,thus addressing a problem with existing approaches,which ignore the semantic differences in different subgraph structures.Furthermore,using a gated recurrent neural network,this model enhances the features of entities by sorting them by distance and extracting the path information in the subgraphs.Experimentally,it is shown that when there are numerous disconnected regions in the subgraph,our model outperforms existing mainstream models in terms of both Area Under the Curve-Precision-Recall(AUC-PR)and Hits@10.Experiments prove that semantic differences in the knowledge graph can be effectively distinguished and verify the effectiveness of this method.展开更多
基金supported in part by the National Natural Science Foundation of China(62176059)supported by The Pennsylvania State University.
文摘Finding suitable initial noise that retains the original image’s information is crucial for image-to-image(I2I)translation using text-to-image(T2I)diffusion models.A common approach is to add random noise directly to the original image,as in SDEdit.However,we have observed that this can result in“semantic discrepancy”issues,wherein T2I diffusion models misinterpret the semantic relationships and generate content not present in the original image.We identify that the noise introduced by SDEdit disrupts the semantic integrity of the image,leading to unintended associations between unrelated regions after U-Net upsampling.Building on the widely-used latent diffusion model,Stable Diffusion,we propose a training-free,plugand-play method to alleviate semantic discrepancy and enhance the fidelity of the translated image.By leveraging the deterministic nature of denoising diffusion implicit models(DDIMs)inversion,we correct the erroneous features and correlations from the original generative process with accurate ones from DDIM inversion.This approach alleviates semantic discrepancy and surpasses recent DDIM-inversion-based methods such as PnP with fewer priors,achieving a speedup of 11.2 times in experiments conducted on COCO,ImageNet,and ImageNet-R datasets across multiple I2I translation tasks.
基金supported by the National Key Research and Development Program of China under Grant No.2022YFB3303203the National Natural Science Foundation of China under Grant No.62272275.
文摘The glinty details from complex microstructures significantly enhance rendering realism.However,the previous methods use high-resolution normal maps to define each micro-geometry,which requires huge memory overhead.This paper observes that many self-similarity materials have independent structural characteristics,which we define as tiny example microstructures.We propose a procedural model to represent microstructures implicitly by performing spatial transformations and spatial distribution on tiny examples.Furthermore,we precompute normal distribution functions(NDFs)by 4D Gaussians for tiny examples and store them in multi-scale NDF maps.Combined with a tiny example based NDF evaluation method,complex glinty surfaces can be rendered simply by texture sampling.The experimental results show that our tiny example based the microstructure rendering method is GPU-friendly,successfully reproducing high-frequency reflection features of different microstructures in real time with low memory and computational overhead.
基金Project (Nos. 40905013 and 60832003) supported by the National Natural Science Foundation of Chinathe Shanghai Natural Science Foundation (No. 11ZR1413400)+1 种基金the Key Scientific Research Project of the Shanghai Education Committee (No. 12YZ007)the Open Project Program of the State Key Lab of CAD&CG,Zhejiang University (No. A1101),China
文摘Deformation is an important research topic in graphics.There are two key issues in mesh deformation:(1) selfintersection and(2) volume preserving.In this paper,we present a new method to construct a vector field for volume-preserving mesh deformation of free-form objects.Volume-preserving is an inherent feature of a curl vector field.Since the field lines of the curl vector field will never intersect with each other,a mesh deformed under a curl vector field can avoid self-intersection between field lines.Designing the vector field based on curl is useful in preserving graphic features and preventing self-intersection.Our proposed algorithm introduces distance field into vector field construction;as a result,the shape of the curl vector field is closely related to the object shape.We define the construction of the curl vector field for translation and rotation and provide some special effects such as twisting and bending.Taking into account the information of the object,this approach can provide easy and intuitive construction for free-form objects.Experimental results show that the approach works effectively in real-time animation.
基金supported by the National Natural Science Foundation of China(No.U19A2059)the 2022 Research Foundation of Chengdu Textile College(No.X22032161).
文摘Currently,most existing inductive relation prediction approaches are based on subgraph structures,with subgraph features extracted using graph neural networks to predict relations.However,subgraphs may contain disconnected regions,which usually represent different semantic ranges.Because not all semantic information about the regions is helpful in relation prediction,we propose a relation prediction model based on a disentangled subgraph structure and implement a feature updating approach based on relevant semantic aggregation.To indirectly achieve the disentangled subgraph structure from a semantic perspective,the mapping of entity features into different semantic spaces and the aggregation of related semantics on each semantic space are updated.The disentangled model can focus on features having higher semantic relevance in the prediction,thus addressing a problem with existing approaches,which ignore the semantic differences in different subgraph structures.Furthermore,using a gated recurrent neural network,this model enhances the features of entities by sorting them by distance and extracting the path information in the subgraphs.Experimentally,it is shown that when there are numerous disconnected regions in the subgraph,our model outperforms existing mainstream models in terms of both Area Under the Curve-Precision-Recall(AUC-PR)and Hits@10.Experiments prove that semantic differences in the knowledge graph can be effectively distinguished and verify the effectiveness of this method.
