Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has...Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has gained more and more attention. In this paper, we mainly focus on point selection problem in terrain simplification using triangulated irregular network. Based on the analysis and comparison of traditional importance measures for each input point, we put forward a new importance measure based on local entropy. The results demonstrate that the local entropy criterion has a better performance than any traditional methods. In addition, it can effectively conquer the 'short-sight' problem associated with the traditional methods.展开更多
This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axi...This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axis based approach, finding corresponding lines using feature based matching method, and 3D line depth computation.展开更多
The increasing scale and complexity of 3D scene design work urge an efficient way to understand the design in multi-disciplinary team and exploit the experiences and underlying knowledge in previous works for reuse.Ho...The increasing scale and complexity of 3D scene design work urge an efficient way to understand the design in multi-disciplinary team and exploit the experiences and underlying knowledge in previous works for reuse.However the previous researches lack of concerning on relationship maintaining and design reuse in knowledge level.We propose a novel semantic driven design reuse system,including a property computation algorithm that enables our system to compute the properties while modeling process to maintain the semantic consistency,and a vertex statics based algorithm that enables the system to recognize scene design pattern as universal semantic model for the same type of scenes.With the universal semantic model,the system conducts the modeling process of future design works by suggestions and constraints on operation.The proposed framework empowers the reuse of 3D scene design on both model level and knowledge level.展开更多
Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and mos...Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and most existing data-driven methods do not differentiate indoor scene examples in terms of quality.In this work,we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling.Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts,via pairwise comparisons based on fuzzy set theory.We also develop a system to use such examples to guide indoor scene modeling using user-specified objects.Specifically,we focus on object groups associated with certain human activities,and define room features to encode the relations between the position and direction of an object group and the room configuration.To perform indoor scene modeling,given an empty room,our system first assesses it in terms of the user-specified object groups,and then places associated objects in the room guided by the assessment results.A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.展开更多
3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes...3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes. However,modeling indoor3 D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey,we provide an overview of recent advances in indoor scene modeling techniques,as well as public datasets and code libraries which can facilitate experiments and evaluation.展开更多
With the support of edge computing,the synergy and collaboration among central cloud,edge cloud,and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture.This integration ...With the support of edge computing,the synergy and collaboration among central cloud,edge cloud,and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture.This integration unlocks the value of data and computational power,presenting significant opportunities for large-scale 3D scene modeling and XR presentation.In this paper,we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture.We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.展开更多
Creating proper 3D models plays an important pole in the development of scene simulation system based on multigen creator and Vega software. However, it is very difficult to construct complex structures by multigen cr...Creating proper 3D models plays an important pole in the development of scene simulation system based on multigen creator and Vega software. However, it is very difficult to construct complex structures by multigen creator. In this paper, an approach is proposed which is utilizing 3dsmax as assistant modeling software. 3D models developed in 3dsmax could be saved in 3ds format and then imported into multigen creator software. The models are revised and then saved in fit format by creator. For reducing model＇s data, simplification strategy is proposed. The problem of constructing complex models in creator is solved smoothly. In the development of digital rocket simulation project, the models constructed by this method have good visual effect, small size, and could be driven by Vega correctly.展开更多
For target detection algorithm under global motion scene, this paper suggests a target detection algorithm based on motion attention fusion model. Firstly, the motion vector field is pre-processed by accumulation and ...For target detection algorithm under global motion scene, this paper suggests a target detection algorithm based on motion attention fusion model. Firstly, the motion vector field is pre-processed by accumulation and median filter;Then, according to the temporal and spatial character of motion vector, the attention fusion model is defined, which is used to detect moving target;Lastly, the edge of video moving target is made exactly by morphologic operation and edge tracking algorithm. The experimental results of different global motion video sequences show the proposed algorithm has a better veracity and speedup than other algorithm.展开更多
目的随着电影内容的复杂化与多样化,电影场景分割成为理解影片结构和支持多媒体应用的重要任务。为提升镜头特征提取和特征关联的有效性,增强镜头序列的上下文感知能力,提出一种混合架构电影场景分割方法(hybrid architecture scene seg...目的随着电影内容的复杂化与多样化,电影场景分割成为理解影片结构和支持多媒体应用的重要任务。为提升镜头特征提取和特征关联的有效性,增强镜头序列的上下文感知能力,提出一种混合架构电影场景分割方法(hybrid architecture scene segmentation network,HASSNet)。方法首先,采用预训练结合微调策略,在大量无场景标签的电影数据上进行无监督预训练,使模型学习有效的镜头特征表示和关联特性,然后在有场景标签的数据上进行微调训练,进一步提升模型性能;其次,模型架构上混合了状态空间模型和自注意力机制模型,分别设计Shot Mamba镜头特征提取模块和Scene Transformer特征关联模块,Shot Mamba通过对镜头图像分块建模提取有效特征表示,Scene Transformer则通过注意力机制对不同镜头特征进行关联建模;最后,采用3种无监督损失函数进行预训练,提升模型在镜头特征提取和关联上的性能,并使用Focal Loss损失函数进行微调,以改善由于类别不平衡导致的精度不足问题。结果实验结果表明,HASSNet在3个数据集上显著提升了场景分割的精度,在典型电影场景分割数据集MovieNet中,与先进的场景分割方法相比,AP(average precision)、mIoU(mean intersection over union)、AUC-ROC(area under the receiver operating characteristic curve)和F1分别提升1.66%、10.54%、0.21%和16.83%,验证了本文提出的HASSNet方法可以有效提升场景边界定位的准确性。结论本文提出的HASSNet方法有效结合了预训练与微调策略,借助混合状态空间模型和自注意力机制模型的特点,增强了镜头的上下文感知能力,使电影场景分割的结果更加准确。展开更多
为提高现有三维建模方法的精度和效率,研究提出基于无人机倾斜摄影与改进样本一致性迭代算法(Sample Consensus with Iterative Algorithm,SAC-IA)算法的三维建模方法,通过结合无人机多视角高分辨率影像采集技术,和改进SAC-IA与迭代最...为提高现有三维建模方法的精度和效率,研究提出基于无人机倾斜摄影与改进样本一致性迭代算法(Sample Consensus with Iterative Algorithm,SAC-IA)算法的三维建模方法,通过结合无人机多视角高分辨率影像采集技术,和改进SAC-IA与迭代最近点算法协同优化点云配准过程,有效提升建筑物三维建模的精度与速度。研究方法在XX古建筑的数字修复实验中成功应用,准确恢复了建筑物的细节,包括屋顶雕花和外立面的裂缝修复,相比传统的地面激光扫描和摄影测量技术,点云配准误差减少了17%~39%。且建模效率较传统方法提高了45%。由此证明,研究方法在提高建筑三维建模精度的同时,也提升了数据采集和处理效率,为复杂建筑物和大规模场景的三维建模提供可靠且高效的解决方案。展开更多
基金This paper is supported by the State Key Laboratory for Image Processing & Intelligent Control (No. TKLJ9903) National Defe
文摘Augmented reality is the merging of synthetic sensory information into a user's perception of a real environment. As one of the most important tasks in augmented scene modeling, terrain simplification research has gained more and more attention. In this paper, we mainly focus on point selection problem in terrain simplification using triangulated irregular network. Based on the analysis and comparison of traditional importance measures for each input point, we put forward a new importance measure based on local entropy. The results demonstrate that the local entropy criterion has a better performance than any traditional methods. In addition, it can effectively conquer the 'short-sight' problem associated with the traditional methods.
文摘This paper presents a method for structured scene modeling using micro stereo vision system with large field of view. The proposed algorithm includes edge detection with Canny detector, line fitting with principle axis based approach, finding corresponding lines using feature based matching method, and 3D line depth computation.
基金the National Natural Science Foundation of China(Nos.61073086 and 70871078)the National High Technology Research and Development Program (863) of China(No.2008AA04Z126)
文摘The increasing scale and complexity of 3D scene design work urge an efficient way to understand the design in multi-disciplinary team and exploit the experiences and underlying knowledge in previous works for reuse.However the previous researches lack of concerning on relationship maintaining and design reuse in knowledge level.We propose a novel semantic driven design reuse system,including a property computation algorithm that enables our system to compute the properties while modeling process to maintain the semantic consistency,and a vertex statics based algorithm that enables the system to recognize scene design pattern as universal semantic model for the same type of scenes.With the universal semantic model,the system conducts the modeling process of future design works by suggestions and constraints on operation.The proposed framework empowers the reuse of 3D scene design on both model level and knowledge level.
基金This work was partially supported by grants from the National Natural Science Foundation of China(61902032)Research Grants Council of the Hong Kong Special Administrative Region,China(CityU 11237116)City University of Hong Kong(7004915).
文摘Well-designed indoor scenes incorporate interior design knowledge,which has been an essential prior for most indoor scene modeling methods.However,the layout qualities of indoor scene datasets are often uneven,and most existing data-driven methods do not differentiate indoor scene examples in terms of quality.In this work,we aim to explore an approach that leverages datasets with differentiated indoor scene examples for indoor scene modeling.Our solution conducts subjective evaluations on lightweight datasets having various room configurations and furniture layouts,via pairwise comparisons based on fuzzy set theory.We also develop a system to use such examples to guide indoor scene modeling using user-specified objects.Specifically,we focus on object groups associated with certain human activities,and define room features to encode the relations between the position and direction of an object group and the room configuration.To perform indoor scene modeling,given an empty room,our system first assesses it in terms of the user-specified object groups,and then places associated objects in the room guided by the assessment results.A series of experimental results and comparisons to state-of-the-art indoor scene synthesis methods are presented to validate the usefulness and effectiveness of our approach.
基金supported by the National Natural Science Foundation of China(Project No.61120106007)Research Grant of Beijing Higher Institution Engineering Research CenterTsinghua University Initiative Scientific Research Program
文摘3D scene modeling has long been a fundamental problem in computer graphics and computer vision. With the popularity of consumer-level RGB-D cameras,there is a growing interest in digitizing real-world indoor 3D scenes. However,modeling indoor3 D scenes remains a challenging problem because of the complex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey,we provide an overview of recent advances in indoor scene modeling techniques,as well as public datasets and code libraries which can facilitate experiments and evaluation.
基金the National Natural Science Foundation of China(U22B2034)the Fundamental Research Funds for the Central Universities(226-2022-00064).
文摘With the support of edge computing,the synergy and collaboration among central cloud,edge cloud,and terminal devices form an integrated computing ecosystem known as the cloud-edge-client architecture.This integration unlocks the value of data and computational power,presenting significant opportunities for large-scale 3D scene modeling and XR presentation.In this paper,we explore the perspectives and highlight new challenges in 3D scene modeling and XR presentation based on point cloud within the cloud-edge-client integrated architecture.We also propose a novel cloud-edge-client integrated technology framework and a demonstration of municipal governance application to address these challenges.
文摘Creating proper 3D models plays an important pole in the development of scene simulation system based on multigen creator and Vega software. However, it is very difficult to construct complex structures by multigen creator. In this paper, an approach is proposed which is utilizing 3dsmax as assistant modeling software. 3D models developed in 3dsmax could be saved in 3ds format and then imported into multigen creator software. The models are revised and then saved in fit format by creator. For reducing model＇s data, simplification strategy is proposed. The problem of constructing complex models in creator is solved smoothly. In the development of digital rocket simulation project, the models constructed by this method have good visual effect, small size, and could be driven by Vega correctly.
文摘For target detection algorithm under global motion scene, this paper suggests a target detection algorithm based on motion attention fusion model. Firstly, the motion vector field is pre-processed by accumulation and median filter;Then, according to the temporal and spatial character of motion vector, the attention fusion model is defined, which is used to detect moving target;Lastly, the edge of video moving target is made exactly by morphologic operation and edge tracking algorithm. The experimental results of different global motion video sequences show the proposed algorithm has a better veracity and speedup than other algorithm.
文摘目的随着电影内容的复杂化与多样化,电影场景分割成为理解影片结构和支持多媒体应用的重要任务。为提升镜头特征提取和特征关联的有效性,增强镜头序列的上下文感知能力,提出一种混合架构电影场景分割方法(hybrid architecture scene segmentation network,HASSNet)。方法首先,采用预训练结合微调策略,在大量无场景标签的电影数据上进行无监督预训练,使模型学习有效的镜头特征表示和关联特性,然后在有场景标签的数据上进行微调训练,进一步提升模型性能;其次,模型架构上混合了状态空间模型和自注意力机制模型,分别设计Shot Mamba镜头特征提取模块和Scene Transformer特征关联模块,Shot Mamba通过对镜头图像分块建模提取有效特征表示,Scene Transformer则通过注意力机制对不同镜头特征进行关联建模;最后,采用3种无监督损失函数进行预训练,提升模型在镜头特征提取和关联上的性能,并使用Focal Loss损失函数进行微调,以改善由于类别不平衡导致的精度不足问题。结果实验结果表明,HASSNet在3个数据集上显著提升了场景分割的精度,在典型电影场景分割数据集MovieNet中,与先进的场景分割方法相比,AP(average precision)、mIoU(mean intersection over union)、AUC-ROC(area under the receiver operating characteristic curve)和F1分别提升1.66%、10.54%、0.21%和16.83%,验证了本文提出的HASSNet方法可以有效提升场景边界定位的准确性。结论本文提出的HASSNet方法有效结合了预训练与微调策略,借助混合状态空间模型和自注意力机制模型的特点,增强了镜头的上下文感知能力,使电影场景分割的结果更加准确。
文摘为提高现有三维建模方法的精度和效率,研究提出基于无人机倾斜摄影与改进样本一致性迭代算法(Sample Consensus with Iterative Algorithm,SAC-IA)算法的三维建模方法,通过结合无人机多视角高分辨率影像采集技术,和改进SAC-IA与迭代最近点算法协同优化点云配准过程,有效提升建筑物三维建模的精度与速度。研究方法在XX古建筑的数字修复实验中成功应用,准确恢复了建筑物的细节,包括屋顶雕花和外立面的裂缝修复,相比传统的地面激光扫描和摄影测量技术,点云配准误差减少了17%~39%。且建模效率较传统方法提高了45%。由此证明,研究方法在提高建筑三维建模精度的同时,也提升了数据采集和处理效率,为复杂建筑物和大规模场景的三维建模提供可靠且高效的解决方案。
