One fundamental problem in computer vision and image processing is modeling the image formation of a camera, i.e., mapping a point in three-dimensional space to its projected position on the camera’s image plane. If ...One fundamental problem in computer vision and image processing is modeling the image formation of a camera, i.e., mapping a point in three-dimensional space to its projected position on the camera’s image plane. If the relationship between the space and the image plane is assumed to be linear, the relationship can be expressed in terms of a transfor-mation matrix and the matrix is often identified by regression. In this paper, we show that the space-to-image relation-ship in a camera can be modeled by a simple neural network. Unlike most other cases employing neural networks, the structure of the network is optimized so as for each link between neurons to have a physical meaning. This makes it possible to effectively initialize link weights and quickly train the network.展开更多
The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments...The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments and virtual sensor models. In the context of this, the present paper documents the development of a sensor model for depth estimation of virtual three-dimensional scenarios. For this purpose, the geometric and algorithmic principles of stereoscopic camera systems are recreated in a virtual form. The model is implemented as a subroutine in the Epic Games Unreal Engine, which is one of the most common Game Engines. Its architecture consists of several independent procedures that enable a local depth estimation, but also a reconstruction of a whole three-dimensional scenery. In addition, a separate programme for calibrating the model is presented. In addition to the basic principles, the architecture and the implementation, this work also documents the evaluation of the model created. It is shown that the model meets specifically defined requirements for real-time capability and the accuracy of the evaluation. Thus, it is suitable for the virtual testing of common algorithms and highly automated driving functions.展开更多
智能汽车是全球汽车产业发展的重要方向,为适应智能汽车实践教学的新需求,设计并搭建了一套智能汽车视频注入实验平台。首先,结合虚拟仿真与车载域控制器,设计了一套虚实融合的实验平台方案;其次,开发了摄像头仿真模型并进行了虚拟标定...智能汽车是全球汽车产业发展的重要方向,为适应智能汽车实践教学的新需求,设计并搭建了一套智能汽车视频注入实验平台。首先,结合虚拟仿真与车载域控制器,设计了一套虚实融合的实验平台方案;其次,开发了摄像头仿真模型并进行了虚拟标定验证,同时构建了基于GMSL2(Gigabit Multimedia Serial Link 2)与CSI-2(Camera Serial Interface 2)协议的摄像头数据链路,实现了虚拟仿真信号与车载域控制器硬件之间的有效交互;将开发环境中训练的深度学习模型进行量化与通用格式转换,并部署至车载域控制器,使其能与虚拟仿真环境实现实时交互;最后,应用建立的实验系统,为学生提供了理论与实践相结合的学习环境,帮助学生深入理解摄像头原理,并掌握多路视频信号从生成、传输到感知算法部署过程。实验结果表明,智能汽车视频注入实验系统能有效提升学生在智能汽车技术领域的实践能力和创新能力。展开更多
Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks a...Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.展开更多
文摘One fundamental problem in computer vision and image processing is modeling the image formation of a camera, i.e., mapping a point in three-dimensional space to its projected position on the camera’s image plane. If the relationship between the space and the image plane is assumed to be linear, the relationship can be expressed in terms of a transfor-mation matrix and the matrix is often identified by regression. In this paper, we show that the space-to-image relation-ship in a camera can be modeled by a simple neural network. Unlike most other cases employing neural networks, the structure of the network is optimized so as for each link between neurons to have a physical meaning. This makes it possible to effectively initialize link weights and quickly train the network.
文摘The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments and virtual sensor models. In the context of this, the present paper documents the development of a sensor model for depth estimation of virtual three-dimensional scenarios. For this purpose, the geometric and algorithmic principles of stereoscopic camera systems are recreated in a virtual form. The model is implemented as a subroutine in the Epic Games Unreal Engine, which is one of the most common Game Engines. Its architecture consists of several independent procedures that enable a local depth estimation, but also a reconstruction of a whole three-dimensional scenery. In addition, a separate programme for calibrating the model is presented. In addition to the basic principles, the architecture and the implementation, this work also documents the evaluation of the model created. It is shown that the model meets specifically defined requirements for real-time capability and the accuracy of the evaluation. Thus, it is suitable for the virtual testing of common algorithms and highly automated driving functions.
文摘智能汽车是全球汽车产业发展的重要方向,为适应智能汽车实践教学的新需求,设计并搭建了一套智能汽车视频注入实验平台。首先,结合虚拟仿真与车载域控制器,设计了一套虚实融合的实验平台方案;其次,开发了摄像头仿真模型并进行了虚拟标定验证,同时构建了基于GMSL2(Gigabit Multimedia Serial Link 2)与CSI-2(Camera Serial Interface 2)协议的摄像头数据链路,实现了虚拟仿真信号与车载域控制器硬件之间的有效交互;将开发环境中训练的深度学习模型进行量化与通用格式转换,并部署至车载域控制器,使其能与虚拟仿真环境实现实时交互;最后,应用建立的实验系统,为学生提供了理论与实践相结合的学习环境,帮助学生深入理解摄像头原理,并掌握多路视频信号从生成、传输到感知算法部署过程。实验结果表明,智能汽车视频注入实验系统能有效提升学生在智能汽车技术领域的实践能力和创新能力。
基金supported by the National Natural Science Foundationof China(61100207)the National Key Technology Research and Development Program of the Ministry of Science and Technology of China(2014BAK14B03)+1 种基金the Fundamental Research Funds for the Central Universities(2013PT132013XZ12)
文摘Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.
