In a controlled indoor environment,line tracking has become the most practical and reliable navigation strategy for autonomous mobile robots.A line tracking robot is a self-mobile machine that can recognize and track ...In a controlled indoor environment,line tracking has become the most practical and reliable navigation strategy for autonomous mobile robots.A line tracking robot is a self-mobile machine that can recognize and track a painted line on thefloor.In general,the path is set and can be visible,such as a black line on a white surface with high contrasting colors.The robot’s path is marked by a distinct line or track,which the robot follows to move.Several scientific contributions from the disciplines of vision and control have been made to mobile robot vision-based navigation.Localization,automated map generation,autonomous navigation and path tracking is all becoming more frequent in vision applications.A visual navigation line tracking robot should detect the line with a camera using an image processing technique.The paper focuses on combining computer vision techniques with a proportional-integral-derivative(PID)control-ler for automatic steering and speed control.A prototype line tracking robot is used to evaluate the proposed control strategy.展开更多
GPS-based technology has served the positioning and navigation industry for decades with outstanding reliability and accuracy.However,limitations for location-based services in indoor scenarios remain where GPS signal...GPS-based technology has served the positioning and navigation industry for decades with outstanding reliability and accuracy.However,limitations for location-based services in indoor scenarios remain where GPS signal reception suffers from severe degradation or even outages.Wi-Fi-based positioning is currently the most popular indoor location solution,with an availability and time-to-first-fix that are significantly better than GPS.However,the achievable position accuracy is only at the level of tens of meters on average depending on database density and signal reception quality.In order to improve positioning accuracy and availability,motion sensors like accelerometers,gyros,and magnetic compasses are increasingly integrated into smart devices.However,their position solutions suffer from the effects of significant accumulative errors.In this paper,a vision-based indoor positioning method is developed to overcome the limitations above.The proposed vision-based system relies on a single camera,widely available on smart phones and tablets.The derivation of the absolute 3D position from 2D snapshots of a single camera requires the use of an external geo-reference database.In this research,a ubiquitous floor plan database has been used to provide accurate geodetic information.Unlike other popular geo-reference databases,the database used in this work can easily be generated with existing resources.The proposed system has been developed as an iOS App and was tested on iPad for various indoor scenarios.The results show that the performance of the proposed system is superior to Wi-Fi-based positioning systems.展开更多
In this paper,we propose a novel vision navigation method based on three-dimensional(3D)reconstruction from real-time image sequences.It adapts 3D reconstruction and terrain matching to establish the correspondence be...In this paper,we propose a novel vision navigation method based on three-dimensional(3D)reconstruction from real-time image sequences.It adapts 3D reconstruction and terrain matching to establish the correspondence between image points and3D space points and the terrain reference(by using a digital elevation map(DEM)).An adaptive weighted orthogonal iterative pose estimation method is employed to calculate the position and attitude angle of the aircraft.Synthesized and real experiments show that the proposed method is capable of providing accurate navigation parameters for a long-endurance flight without using a global positioning system or an inertial navigation system(INS).Moreover,it can be combined with an INS to achieve an improved navigation result.展开更多
A dead reckoning system and a vision navigation system are proposed for use in a new integrated system for robot navigation. Since the dead reckoning system uses a recurrence algorithm to determine the position, the p...A dead reckoning system and a vision navigation system are proposed for use in a new integrated system for robot navigation. Since the dead reckoning system uses a recurrence algorithm to determine the position, the position will be divergent in two horizontal directions with time increasing. In order to overcome this defect, a vision navigation system is used to periodically correct the dead reckoning system, and a kalman filter is used to estimate the errors of navigation and the unknown biases of sensors, and precise position and heading estimations are obtained by updating navigation errors and sensors’ biases. It is concluded from the simulation results that all the navigation parameters can be obtained through kalman filtering, and the integrated navigation system proposed for robot navigation can be used in an actual robot working in a laboratory. The measurement noise analysis shows that with the distance between beacon and robot increasing, the measurement noise will increase, and in order to achieve a proper estimation accuracy, the distance should not be too great.展开更多
For the improvement of accuracy and better fault-tolerant performance, a global position system （GPS）/vision navigation （VISNAV） integrated relative navigation and attitude determination approach is presented for ...For the improvement of accuracy and better fault-tolerant performance, a global position system （GPS）/vision navigation （VISNAV） integrated relative navigation and attitude determination approach is presented for ultra-close spacecraft formation flying. Onboard GPS and VISNAV system are adopted and a federal Kalman filter architecture is used for the total navigation system design. Simulation results indicate that the integrated system can provide a total improvement of relative navigation and attitude estimation performance in accuracy and fault-tolerance.展开更多
Obtaining absolute pose based on pre-loaded satellite images is one of the important means of autonomous navigation for small Unmanned Aerial Vehicles(UAVs)in Global Navigation Satellite System(GNSS)denied environment...Obtaining absolute pose based on pre-loaded satellite images is one of the important means of autonomous navigation for small Unmanned Aerial Vehicles(UAVs)in Global Navigation Satellite System(GNSS)denied environments.Most of the previous works have tended to build Convolutional Neural Networks(CNNs)to extract features and then directly regress the pose,which will fail when solving the challenges caused by the huge viewpoint and size differences between“UAV-satellite”image pairs in real-world scenarios.Therefore,this paper proposes a probability distribution/regression integrated deep model with the attention-guided triple fusion mechanism,which estimates discrete distributions in pose space and three-dimensional vectors in translation space.In order to overcome the shortage of the relevant dataset,this paper simulates image datasets captured by UAVs with forward-facing cameras during target searching and autonomous attacking.The effectiveness,superiority,and robustness of the proposed method are verified by simulated datasets and flight tests.展开更多
Due to the influence of complex working environment and artificial factors,it is easy to cause crop up over or less tillage problem when straw returning machine is working in paddy field.A new method for path detectio...Due to the influence of complex working environment and artificial factors,it is easy to cause crop up over or less tillage problem when straw returning machine is working in paddy field.A new method for path detection suitable for rice,rape and wheat high crop stubble tilling environments was proposed.First the distribution characteristics of rice,rape and wheat high crop stubble images in paddy field based on RGB color model were analyzed,and rice,the color images of rape and wheat high crop stubble were converted into gray ones using custom factor combination R+G-2B;Then,the gray images of rice,rape and wheat high crop stubble were segmented from soil background by means of luminance mean texture descriptor;Next,the binary image through custom shear-binary-image algorithm was cut to remove big noise blobs in high crop stubble’s tilled area;Finally,navigation path from navigation points by using the least square method was derived.The experimental results indicated that the navigation path detection algorithm was fast and effective to obtain navigation path in rice,rape and wheat high crop stubble tilling environments with up to 96.7% of segmentation accuracy within 0.6 s of processing time.展开更多
As humans,we can naturally break down a task into individual steps in our daily lives and we are able to provide feedback or dynamically adjust the plan when encountering obstacles.Similarly,our aim is to facilitate a...As humans,we can naturally break down a task into individual steps in our daily lives and we are able to provide feedback or dynamically adjust the plan when encountering obstacles.Similarly,our aim is to facilitate agents in comprehending and carrying out natural language instructions in a more efficient and cost-effective manner.For example,in Vision–Language Navigation(VLN)tasks,the agent needs to understand instructions such as“go to the table by the fridge”.This understanding allows the agent to navigate to the table and infer that the destination is likely to be in the kitchen.The traditional VLN approach mainly involves training models using a large number of labeled datasets for task planning in unseen environments.However,manual labeling incurs a high cost for this approach.Considering that large language models(LLMs)already possess extensive commonsense knowledge during pretraining,some researchers have started using LLMs as decision modules in embodied tasks,although this approach shows the LLMs’reasoning ability to plan a logical sequence of subtasks based on global information.However,executing subtasks often encounters issues,such as obstacles that hinder progress and alterations in the state of the target object.Even one mistake can cause the subsequent tasks to fail,which makes it challenging to complete the instructions through a single plan.Therefore,we propose a new approach—C(Correction)and P(Planning)with M(Memory)I(Integration)—that centered on an LLM for embodied tasks.In more detail,the auxiliary modules of the CPMI facilitate dynamic planning by the LLM-centric planner.These modules provide the agent with memory and generalized experience mechanisms to fully utilize the LLM capabilities,allowing it to improve its performance during execution.Finally,the experimental results on public datasets demonstrate that we achieve the best performance in the few-shot scenario,improving the efficiency of the successive task while increasing the success rate.展开更多
基金funding from the researchers supporting project number(RSP2022R474)King Saud University,Riyadh,Saudi Arabia.
文摘In a controlled indoor environment,line tracking has become the most practical and reliable navigation strategy for autonomous mobile robots.A line tracking robot is a self-mobile machine that can recognize and track a painted line on thefloor.In general,the path is set and can be visible,such as a black line on a white surface with high contrasting colors.The robot’s path is marked by a distinct line or track,which the robot follows to move.Several scientific contributions from the disciplines of vision and control have been made to mobile robot vision-based navigation.Localization,automated map generation,autonomous navigation and path tracking is all becoming more frequent in vision applications.A visual navigation line tracking robot should detect the line with a camera using an image processing technique.The paper focuses on combining computer vision techniques with a proportional-integral-derivative(PID)control-ler for automatic steering and speed control.A prototype line tracking robot is used to evaluate the proposed control strategy.
基金This research is financially supported by the Natural Sciences and Engineering Research Council(NSERC)of Canada and Tecterra.
文摘GPS-based technology has served the positioning and navigation industry for decades with outstanding reliability and accuracy.However,limitations for location-based services in indoor scenarios remain where GPS signal reception suffers from severe degradation or even outages.Wi-Fi-based positioning is currently the most popular indoor location solution,with an availability and time-to-first-fix that are significantly better than GPS.However,the achievable position accuracy is only at the level of tens of meters on average depending on database density and signal reception quality.In order to improve positioning accuracy and availability,motion sensors like accelerometers,gyros,and magnetic compasses are increasingly integrated into smart devices.However,their position solutions suffer from the effects of significant accumulative errors.In this paper,a vision-based indoor positioning method is developed to overcome the limitations above.The proposed vision-based system relies on a single camera,widely available on smart phones and tablets.The derivation of the absolute 3D position from 2D snapshots of a single camera requires the use of an external geo-reference database.In this research,a ubiquitous floor plan database has been used to provide accurate geodetic information.Unlike other popular geo-reference databases,the database used in this work can easily be generated with existing resources.The proposed system has been developed as an iOS App and was tested on iPad for various indoor scenarios.The results show that the performance of the proposed system is superior to Wi-Fi-based positioning systems.
基金supported by the National Basic Research Program of China("973"Project)(Grant No.2013CB733100)
文摘In this paper,we propose a novel vision navigation method based on three-dimensional(3D)reconstruction from real-time image sequences.It adapts 3D reconstruction and terrain matching to establish the correspondence between image points and3D space points and the terrain reference(by using a digital elevation map(DEM)).An adaptive weighted orthogonal iterative pose estimation method is employed to calculate the position and attitude angle of the aircraft.Synthesized and real experiments show that the proposed method is capable of providing accurate navigation parameters for a long-endurance flight without using a global positioning system or an inertial navigation system(INS).Moreover,it can be combined with an INS to achieve an improved navigation result.
文摘A dead reckoning system and a vision navigation system are proposed for use in a new integrated system for robot navigation. Since the dead reckoning system uses a recurrence algorithm to determine the position, the position will be divergent in two horizontal directions with time increasing. In order to overcome this defect, a vision navigation system is used to periodically correct the dead reckoning system, and a kalman filter is used to estimate the errors of navigation and the unknown biases of sensors, and precise position and heading estimations are obtained by updating navigation errors and sensors’ biases. It is concluded from the simulation results that all the navigation parameters can be obtained through kalman filtering, and the integrated navigation system proposed for robot navigation can be used in an actual robot working in a laboratory. The measurement noise analysis shows that with the distance between beacon and robot increasing, the measurement noise will increase, and in order to achieve a proper estimation accuracy, the distance should not be too great.
文摘For the improvement of accuracy and better fault-tolerant performance, a global position system （GPS）/vision navigation （VISNAV） integrated relative navigation and attitude determination approach is presented for ultra-close spacecraft formation flying. Onboard GPS and VISNAV system are adopted and a federal Kalman filter architecture is used for the total navigation system design. Simulation results indicate that the integrated system can provide a total improvement of relative navigation and attitude estimation performance in accuracy and fault-tolerance.
基金supported by the National Natural Science Foundation of China(No.61973033)the Chongqing Natural Science Foundation,China(No.cstc2021jcyjmsxmX0737).
文摘Obtaining absolute pose based on pre-loaded satellite images is one of the important means of autonomous navigation for small Unmanned Aerial Vehicles(UAVs)in Global Navigation Satellite System(GNSS)denied environments.Most of the previous works have tended to build Convolutional Neural Networks(CNNs)to extract features and then directly regress the pose,which will fail when solving the challenges caused by the huge viewpoint and size differences between“UAV-satellite”image pairs in real-world scenarios.Therefore,this paper proposes a probability distribution/regression integrated deep model with the attention-guided triple fusion mechanism,which estimates discrete distributions in pose space and three-dimensional vectors in translation space.In order to overcome the shortage of the relevant dataset,this paper simulates image datasets captured by UAVs with forward-facing cameras during target searching and autonomous attacking.The effectiveness,superiority,and robustness of the proposed method are verified by simulated datasets and flight tests.
基金financially supported by the Special Fund for Agro-scientific Research in the Public Interest(No.201203059)the Natural Science Foundation of China(No.51275196).
文摘Due to the influence of complex working environment and artificial factors,it is easy to cause crop up over or less tillage problem when straw returning machine is working in paddy field.A new method for path detection suitable for rice,rape and wheat high crop stubble tilling environments was proposed.First the distribution characteristics of rice,rape and wheat high crop stubble images in paddy field based on RGB color model were analyzed,and rice,the color images of rape and wheat high crop stubble were converted into gray ones using custom factor combination R+G-2B;Then,the gray images of rice,rape and wheat high crop stubble were segmented from soil background by means of luminance mean texture descriptor;Next,the binary image through custom shear-binary-image algorithm was cut to remove big noise blobs in high crop stubble’s tilled area;Finally,navigation path from navigation points by using the least square method was derived.The experimental results indicated that the navigation path detection algorithm was fast and effective to obtain navigation path in rice,rape and wheat high crop stubble tilling environments with up to 96.7% of segmentation accuracy within 0.6 s of processing time.
基金supported by the Program of Natural Science Foundation of Shanghai(No.23ZR1422800).
文摘As humans,we can naturally break down a task into individual steps in our daily lives and we are able to provide feedback or dynamically adjust the plan when encountering obstacles.Similarly,our aim is to facilitate agents in comprehending and carrying out natural language instructions in a more efficient and cost-effective manner.For example,in Vision–Language Navigation(VLN)tasks,the agent needs to understand instructions such as“go to the table by the fridge”.This understanding allows the agent to navigate to the table and infer that the destination is likely to be in the kitchen.The traditional VLN approach mainly involves training models using a large number of labeled datasets for task planning in unseen environments.However,manual labeling incurs a high cost for this approach.Considering that large language models(LLMs)already possess extensive commonsense knowledge during pretraining,some researchers have started using LLMs as decision modules in embodied tasks,although this approach shows the LLMs’reasoning ability to plan a logical sequence of subtasks based on global information.However,executing subtasks often encounters issues,such as obstacles that hinder progress and alterations in the state of the target object.Even one mistake can cause the subsequent tasks to fail,which makes it challenging to complete the instructions through a single plan.Therefore,we propose a new approach—C(Correction)and P(Planning)with M(Memory)I(Integration)—that centered on an LLM for embodied tasks.In more detail,the auxiliary modules of the CPMI facilitate dynamic planning by the LLM-centric planner.These modules provide the agent with memory and generalized experience mechanisms to fully utilize the LLM capabilities,allowing it to improve its performance during execution.Finally,the experimental results on public datasets demonstrate that we achieve the best performance in the few-shot scenario,improving the efficiency of the successive task while increasing the success rate.
