Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks a...Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.展开更多
A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cam...A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.展开更多
The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments...The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments and virtual sensor models. In the context of this, the present paper documents the development of a sensor model for depth estimation of virtual three-dimensional scenarios. For this purpose, the geometric and algorithmic principles of stereoscopic camera systems are recreated in a virtual form. The model is implemented as a subroutine in the Epic Games Unreal Engine, which is one of the most common Game Engines. Its architecture consists of several independent procedures that enable a local depth estimation, but also a reconstruction of a whole three-dimensional scenery. In addition, a separate programme for calibrating the model is presented. In addition to the basic principles, the architecture and the implementation, this work also documents the evaluation of the model created. It is shown that the model meets specifically defined requirements for real-time capability and the accuracy of the evaluation. Thus, it is suitable for the virtual testing of common algorithms and highly automated driving functions.展开更多
First,the constitution of traditional visual sensor is presented.The linear camera model is introduced and the transform matrix between the image coordinate system and the world coordinate system is established.The ba...First,the constitution of traditional visual sensor is presented.The linear camera model is introduced and the transform matrix between the image coordinate system and the world coordinate system is established.The basic principle of camera calibration is expatiated based on the linear camera model. On the basis of a detailed analysis of camera model,a new-style visual sensor for measurement is advanced.It can realize the real time control of the zoom of camera lens by step motor according to the size of objects.Moreover,re-calibration could be avoided and the transform matrix can be acquired by calculating,which can greatly simplify camera calibration process and save the time. Clearer images are gained,so the measurement system precision could be greatly improved.The basic structure of the visual sensor zoom is introduced,including the constitute mode and the movement rule of the fixed former part,zoom part,compensatory part and the fixed latter port.The realization method of zoom controlled by step motor is introduced. Finally,the constitution of the new-style visual sensor is introduced,including hardware and software.The hardware system is composed by manual zoom,CCD camera,image card,gearing,step motor,step motor driver and computer.The realization of software is introduced,including the composed module of software and the workflow of measurement system in the form of structured block diagram.展开更多
Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision...Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision-based algorithms. In this paper we propose a novel approach to address the problem of rolling shutter distortion in aerial imaging. A mathematical model is established by the coordinate transformation method. It can directly calculate the pixel distortion when an aerial camera is imaging at arbitrary gesture angles.Then all pixel distortions form a distortion map over the whole CMOS array and the map is exploited in the image rectification process incorporating reverse projection. The error analysis indicates that within the margin of measuring errors,the final calculation error of our model is less than 1/2 pixel. The experimental results show that our approach yields good rectification performance in a series of images with different distortions. We demonstrate that our method outperforms other vision-based algorithms in terms of the computational complexity,which makes it more suitable for aerial real-time imaging.展开更多
The presence of increased memory and computational power in imaging sensor networks attracts researchers to exploit image processing algorithms on distributed memory and computational power. In this paper, a typical p...The presence of increased memory and computational power in imaging sensor networks attracts researchers to exploit image processing algorithms on distributed memory and computational power. In this paper, a typical perimeter is investigated with a number of sensors placed to form an image sensor network for the purpose of content based distributed image search. Image search algorithm is used to enable distributed content based image search within each sensor node. The energy model is presented to calculate energy efficiency for various cases of image search and transmission. The simulations are carried out based on consideration of continuous monitoring or event driven activity on the perimeter. The simulation setups consider distributed image processing on sensor nodes and results show that energy saving is significant if search algorithms are embedded in image sensor nodes and image processing is distributed across sensor nodes. The tradeoff between sensor life time, distributed image search and network deployed cost is also investigated.展开更多
Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused mor...Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused more and more research interests. In this paper a new method based on multiple view geometry is presented for spatial side information generation of uncalibrated video sensor network. Trifocal tensor encapsulates all the geometric relations among three views that are independent of scene structure; it can be computed from image correspondences alone without requiring knowledge of the motion or calibration. Simulation results show that trifocal tensor-based spatial side information improves the rate-distortion performance over motion compensation based interpolation side information by a maximum gap of around 2dB. Then fusion merges the different side information （temporal and spatial） in order to improve the quality of the final one. Simulation results show that the rate-distortion gains about 0.4 dB.展开更多
Background Depth sensor is an essential element in virtual and augmented reality devices to digitalize users'environment in real time.The current popular technologies include the stereo,structured light,and Time-o...Background Depth sensor is an essential element in virtual and augmented reality devices to digitalize users'environment in real time.The current popular technologies include the stereo,structured light,and Time-of-Flight(ToF).The stereo and structured light method require a baseline separation between multiple sensors for depth sensing,and both suffer from a limited measurement range.The ToF depth sensors have the largest depth range but the lowest depth map resolution.To overcome these problems,we propose a co-axial depth map sensor which is potentially more compact and cost-effective than conventional structured light depth cameras.Meanwhile,it can extend the depth range while maintaining a high depth map resolution.Also,it provides a high-resolution 2 D image along with the 3 D depth map.Methods This depth sensor is constructed with a projection path and an imaging path.Those two paths are combined by a beamsplitter for a co-axial design.In the projection path,a cylindrical lens is inserted to add extra power in one direction which creates an astigmatic pattern.For depth measurement,the astigmatic pattern is projected onto the test scene,and then the depth information can be calculated from the contrast change of the reflected pattern image in two orthogonal directions.To extend the depth measurement range,we use an electronically focus tunable lens at the system stop and tune the power to implement an extended depth range without compromising depth resolution.Results In the depth measurement simulation,we project a resolution target onto a white screen which is moving along the optical axis and then tune the focus tunable lens power for three depth measurement subranges,namely,near,middle and far.In each sub-range,as the test screen moves away from the depth sensor,the horizontal contrast keeps increasing while the vertical contrast keeps decreasing in the reflected image.Therefore,the depth information can be obtained by computing the contrast ratio between features in orthogonal directions.Conclusions The proposed depth map sensor could implement depth measurement for an extended depth range with a co-axial design.展开更多
This paper aims to develop an automatic miscalibration detection and correction framework to maintain accurate calibration of LiDAR and camera for autonomous vehicle after the sensor drift.First,a monitoring algorithm...This paper aims to develop an automatic miscalibration detection and correction framework to maintain accurate calibration of LiDAR and camera for autonomous vehicle after the sensor drift.First,a monitoring algorithm that can continuously detect the miscalibration in each frame is designed,leveraging the rotational motion each individual sensor observes.Then,as sensor drift occurs,the projection constraints between visual feature points and LiDAR 3-D points are used to compute the scaled camera motion,which is further utilized to align the drifted LiDAR scan with the camera image.Finally,the proposed method is sufficiently compared with two representative approaches in the online experiments with varying levels of random drift,then the method is further extended to the offline calibration experiment and is demonstrated by a comparison with two existing benchmark methods.展开更多
Wireless sensor networks have been identified as one of the key technologies for the 21st century. In order to overcome their limitations such as fault tolerance and conservation of energy, we propose a middleware sol...Wireless sensor networks have been identified as one of the key technologies for the 21st century. In order to overcome their limitations such as fault tolerance and conservation of energy, we propose a middleware solution, In-Motes. In-Motes stands as a fault tolerant platform for deploying and monitoring applications in real time offers a number of possibilities for the end user giving him in parallel the freedom to experiment with various parameters, in an effort the deployed applications to run in an energy efficient manner inside the network. The proposed scheme is evaluated through the In-Motes EYE application, aiming to test its merits under real time conditions. In-Motes EYE application which is an agent based real time In-Motes application developed for sensing acceleration variations in an environment. The application was tested in a prototype area, road alike, for a period of four months.展开更多
In this study, we analyzed the swing motions of more experienced practitioner and new practitioner of iaido players by using tri-axial acceleration sensor and gyro sensor. Iaido is a modern Japanese martial art/sport....In this study, we analyzed the swing motions of more experienced practitioner and new practitioner of iaido players by using tri-axial acceleration sensor and gyro sensor. Iaido is a modern Japanese martial art/sport. In this way, the acceleration and gyro sensor measurement enabled detailed motion information at the installation points to be displayed in a short time, thus making it possible to easily extract the objective problems. Although it was not possible to confirm by the acceleration and angular velocity measurements the detailed motion of the entire body as obtained in the 2D motion analysis with a high-speed camera, it was confirmed that the acceleration and gyro sensor is an evaluation means that can be installed easily and can provide the exercise information in a short time as an objective index.展开更多
Professional truck drivers are an essential part of transportation in keeping the global economy alive and commercial products moving. In order to increase productivity and improve safety, an increasing amount of auto...Professional truck drivers are an essential part of transportation in keeping the global economy alive and commercial products moving. In order to increase productivity and improve safety, an increasing amount of automation is implemented in modern trucks. Transition to automated heavy good vehicles is intended to make trucks accident-free and, on the other hand, more comfortable to drive. This motivates the automotive industry to bring more embedded ICT into their vehicles in the future. An avenue towards autonomous vehicles requires robust environmental perception and driver monitoring technologies to be introduced. This is the main motivation behind the DESERVE project. This is the study of sensor technology trials in order to minimize blind spots around the truck and, on the other hand, keep the driver’s vigilance at a sufficiently high level. The outcomes are two innovative truck demonstrations: one R & D study for bringing equipment to production in the future and one implementation to the driver training vehicle. The earlier experiments include both driver monitoring technology which works at a 60% - 80% accuracy level and environment perception (stereo and thermal cameras) whose performance rates are 70% - 100%. The results are not sufficient for autonomous vehicles, but are a step forward, since they are in-line even if moved from the lab to real automotive implementations.展开更多
基金supported by the National Natural Science Foundationof China(61100207)the National Key Technology Research and Development Program of the Ministry of Science and Technology of China(2014BAK14B03)+1 种基金the Fundamental Research Funds for the Central Universities(2013PT132013XZ12)
文摘Most sensors or cameras discussed in the sensor network community are usually 3D homogeneous, even though their2 D coverage areas in the ground plane are heterogeneous. Meanwhile, observed objects of camera networks are usually simplified as 2D points in previous literature. However in actual application scenes, not only cameras are always heterogeneous with different height and action radiuses, but also the observed objects are with 3D features（i.e., height）. This paper presents a sensor planning formulation addressing the efficiency enhancement of visual tracking in 3D heterogeneous camera networks that track and detect people traversing a region. The problem of sensor planning consists of three issues：（i） how to model the 3D heterogeneous cameras;（ii） how to rank the visibility, which ensures that the object of interest is visible in a camera＇s field of view;（iii） how to reconfigure the 3D viewing orientations of the cameras. This paper studies the geometric properties of 3D heterogeneous camera networks and addresses an evaluation formulation to rank the visibility of observed objects. Then a sensor planning method is proposed to improve the efficiency of visual tracking. Finally, the numerical results show that the proposed method can improve the tracking performance of the system compared to the conventional strategies.
基金Project supported by the National Natural Science Foundation of China (No. 60772134)the Innovation Foundation of Xidian University,China (No. Chuang 05018)
文摘A novel color compensation method for multi-view video coding (MVC) is proposed, which efficiently exploits the inter-view dependencies between views with the existence of color mismatch caused by the diversity of cameras. A color compensation model is developed in RGB channels and then extended to YCbCr channels for practical use. A modified inter-view reference picture is constructed based on the color compensation model, which is more similar to the coding picture than the original inter-view reference picture. Moreover, the color compensation factors can be derived in both encoder and decoder, therefore no additional data need to be transmitted to the decoder. The experimental results show that the proposed method improves the coding efficiency of MVC and maintains good subjective quality.
文摘The need for efficient and reproducible development processes for sensor and perception systems is growing with their increased use in modern vehicles. Such processes can be achieved by using virtual test environments and virtual sensor models. In the context of this, the present paper documents the development of a sensor model for depth estimation of virtual three-dimensional scenarios. For this purpose, the geometric and algorithmic principles of stereoscopic camera systems are recreated in a virtual form. The model is implemented as a subroutine in the Epic Games Unreal Engine, which is one of the most common Game Engines. Its architecture consists of several independent procedures that enable a local depth estimation, but also a reconstruction of a whole three-dimensional scenery. In addition, a separate programme for calibrating the model is presented. In addition to the basic principles, the architecture and the implementation, this work also documents the evaluation of the model created. It is shown that the model meets specifically defined requirements for real-time capability and the accuracy of the evaluation. Thus, it is suitable for the virtual testing of common algorithms and highly automated driving functions.
文摘First,the constitution of traditional visual sensor is presented.The linear camera model is introduced and the transform matrix between the image coordinate system and the world coordinate system is established.The basic principle of camera calibration is expatiated based on the linear camera model. On the basis of a detailed analysis of camera model,a new-style visual sensor for measurement is advanced.It can realize the real time control of the zoom of camera lens by step motor according to the size of objects.Moreover,re-calibration could be avoided and the transform matrix can be acquired by calculating,which can greatly simplify camera calibration process and save the time. Clearer images are gained,so the measurement system precision could be greatly improved.The basic structure of the visual sensor zoom is introduced,including the constitute mode and the movement rule of the fixed former part,zoom part,compensatory part and the fixed latter port.The realization method of zoom controlled by step motor is introduced. Finally,the constitution of the new-style visual sensor is introduced,including hardware and software.The hardware system is composed by manual zoom,CCD camera,image card,gearing,step motor,step motor driver and computer.The realization of software is introduced,including the composed module of software and the workflow of measurement system in the form of structured block diagram.
基金Sponsored by the National Natural Science Foundation of China(Grant No.60902067)the Foundation for Science & Technology Research Project of Jilin Province(Grant No.11ZDGG001)
文摘Due to the electronic rolling shutter, high-speed Complementary Metal-Oxide Semiconductor( CMOS) aerial cameras are generally subject to geometric distortions,which cannot be perfectly corrected by conventional vision-based algorithms. In this paper we propose a novel approach to address the problem of rolling shutter distortion in aerial imaging. A mathematical model is established by the coordinate transformation method. It can directly calculate the pixel distortion when an aerial camera is imaging at arbitrary gesture angles.Then all pixel distortions form a distortion map over the whole CMOS array and the map is exploited in the image rectification process incorporating reverse projection. The error analysis indicates that within the margin of measuring errors,the final calculation error of our model is less than 1/2 pixel. The experimental results show that our approach yields good rectification performance in a series of images with different distortions. We demonstrate that our method outperforms other vision-based algorithms in terms of the computational complexity,which makes it more suitable for aerial real-time imaging.
文摘The presence of increased memory and computational power in imaging sensor networks attracts researchers to exploit image processing algorithms on distributed memory and computational power. In this paper, a typical perimeter is investigated with a number of sensors placed to form an image sensor network for the purpose of content based distributed image search. Image search algorithm is used to enable distributed content based image search within each sensor node. The energy model is presented to calculate energy efficiency for various cases of image search and transmission. The simulations are carried out based on consideration of continuous monitoring or event driven activity on the perimeter. The simulation setups consider distributed image processing on sensor nodes and results show that energy saving is significant if search algorithms are embedded in image sensor nodes and image processing is distributed across sensor nodes. The tradeoff between sensor life time, distributed image search and network deployed cost is also investigated.
文摘Distributed video coding （DVC） is a new video coding approach based on Wyner-Ziv theorem. The novel uplink-friendly DVC, which offers low-complexity, low-power consuming, and low-cost video encoding, has aroused more and more research interests. In this paper a new method based on multiple view geometry is presented for spatial side information generation of uncalibrated video sensor network. Trifocal tensor encapsulates all the geometric relations among three views that are independent of scene structure; it can be computed from image correspondences alone without requiring knowledge of the motion or calibration. Simulation results show that trifocal tensor-based spatial side information improves the rate-distortion performance over motion compensation based interpolation side information by a maximum gap of around 2dB. Then fusion merges the different side information （temporal and spatial） in order to improve the quality of the final one. Simulation results show that the rate-distortion gains about 0.4 dB.
文摘Background Depth sensor is an essential element in virtual and augmented reality devices to digitalize users'environment in real time.The current popular technologies include the stereo,structured light,and Time-of-Flight(ToF).The stereo and structured light method require a baseline separation between multiple sensors for depth sensing,and both suffer from a limited measurement range.The ToF depth sensors have the largest depth range but the lowest depth map resolution.To overcome these problems,we propose a co-axial depth map sensor which is potentially more compact and cost-effective than conventional structured light depth cameras.Meanwhile,it can extend the depth range while maintaining a high depth map resolution.Also,it provides a high-resolution 2 D image along with the 3 D depth map.Methods This depth sensor is constructed with a projection path and an imaging path.Those two paths are combined by a beamsplitter for a co-axial design.In the projection path,a cylindrical lens is inserted to add extra power in one direction which creates an astigmatic pattern.For depth measurement,the astigmatic pattern is projected onto the test scene,and then the depth information can be calculated from the contrast change of the reflected pattern image in two orthogonal directions.To extend the depth measurement range,we use an electronically focus tunable lens at the system stop and tune the power to implement an extended depth range without compromising depth resolution.Results In the depth measurement simulation,we project a resolution target onto a white screen which is moving along the optical axis and then tune the focus tunable lens power for three depth measurement subranges,namely,near,middle and far.In each sub-range,as the test screen moves away from the depth sensor,the horizontal contrast keeps increasing while the vertical contrast keeps decreasing in the reflected image.Therefore,the depth information can be obtained by computing the contrast ratio between features in orthogonal directions.Conclusions The proposed depth map sensor could implement depth measurement for an extended depth range with a co-axial design.
基金Supported by National Natural Science Foundation of China(Grant Nos.52025121,52394263)National Key R&D Plan of China(Grant No.2023YFD2000301).
文摘This paper aims to develop an automatic miscalibration detection and correction framework to maintain accurate calibration of LiDAR and camera for autonomous vehicle after the sensor drift.First,a monitoring algorithm that can continuously detect the miscalibration in each frame is designed,leveraging the rotational motion each individual sensor observes.Then,as sensor drift occurs,the projection constraints between visual feature points and LiDAR 3-D points are used to compute the scaled camera motion,which is further utilized to align the drifted LiDAR scan with the camera image.Finally,the proposed method is sufficiently compared with two representative approaches in the online experiments with varying levels of random drift,then the method is further extended to the offline calibration experiment and is demonstrated by a comparison with two existing benchmark methods.
文摘Wireless sensor networks have been identified as one of the key technologies for the 21st century. In order to overcome their limitations such as fault tolerance and conservation of energy, we propose a middleware solution, In-Motes. In-Motes stands as a fault tolerant platform for deploying and monitoring applications in real time offers a number of possibilities for the end user giving him in parallel the freedom to experiment with various parameters, in an effort the deployed applications to run in an energy efficient manner inside the network. The proposed scheme is evaluated through the In-Motes EYE application, aiming to test its merits under real time conditions. In-Motes EYE application which is an agent based real time In-Motes application developed for sensing acceleration variations in an environment. The application was tested in a prototype area, road alike, for a period of four months.
文摘In this study, we analyzed the swing motions of more experienced practitioner and new practitioner of iaido players by using tri-axial acceleration sensor and gyro sensor. Iaido is a modern Japanese martial art/sport. In this way, the acceleration and gyro sensor measurement enabled detailed motion information at the installation points to be displayed in a short time, thus making it possible to easily extract the objective problems. Although it was not possible to confirm by the acceleration and angular velocity measurements the detailed motion of the entire body as obtained in the 2D motion analysis with a high-speed camera, it was confirmed that the acceleration and gyro sensor is an evaluation means that can be installed easily and can provide the exercise information in a short time as an objective index.
基金European Commission under the ECSEL Joint Undertaking and TEKES–the Finnish Funding Agency for Innovation
文摘Professional truck drivers are an essential part of transportation in keeping the global economy alive and commercial products moving. In order to increase productivity and improve safety, an increasing amount of automation is implemented in modern trucks. Transition to automated heavy good vehicles is intended to make trucks accident-free and, on the other hand, more comfortable to drive. This motivates the automotive industry to bring more embedded ICT into their vehicles in the future. An avenue towards autonomous vehicles requires robust environmental perception and driver monitoring technologies to be introduced. This is the main motivation behind the DESERVE project. This is the study of sensor technology trials in order to minimize blind spots around the truck and, on the other hand, keep the driver’s vigilance at a sufficiently high level. The outcomes are two innovative truck demonstrations: one R & D study for bringing equipment to production in the future and one implementation to the driver training vehicle. The earlier experiments include both driver monitoring technology which works at a 60% - 80% accuracy level and environment perception (stereo and thermal cameras) whose performance rates are 70% - 100%. The results are not sufficient for autonomous vehicles, but are a step forward, since they are in-line even if moved from the lab to real automotive implementations.
